[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786207919384338-8288-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 16:51:59 dist-test-slave-6zbq krb5kdc[8306](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 16:51:59 dist-test-slave-6zbq krb5kdc[8306](info): set up 2 sockets
Aug 08 16:51:59 dist-test-slave-6zbq krb5kdc[8306](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260808 16:52:00.565299 8288 mini_kdc.cc:121] Time spent starting KDC: real 1.146s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786207919384338-8288-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786207919384338-8288-0/krb5kdc/test-user.keytab.
Aug 08 16:52:00 dist-test-slave-6zbq krb5kdc[8306](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207920, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T16:52:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T16:52:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.24.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
I20260808 16:52:00.672657 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786207919384338-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786207919384338-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786207919384338-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:44065
--webserver_interface=127.8.24.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:43829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:44065
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:52:00.771889 8326 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:52:00.772083 8326 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:52:00.772121 8326 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:52:00.773303 8326 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 16:52:00.773343 8326 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:52:00.773362 8326 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:52:00.773377 8326 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 16:52:00.773402 8326 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 16:52:00.775179 8326 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:43829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786207919384338-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786207919384338-8288-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:44065
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:44065
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.24.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8326
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786207919384338-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:52:00.775565 8326 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:52:00.775825 8326 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:52:00.782176 8336 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:00.782222 8335 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:00.782496 8326 server_base.cc:1034] running on GCE node
W20260808 16:52:00.783011 8338 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:00.783279 8326 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:52:00.783643 8326 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:52:00.784792 8326 hybrid_clock.cc:648] HybridClock initialized: now 1786207920784794 us; error 43 us; skew 500 ppm
Aug 08 16:52:00 dist-test-slave-6zbq krb5kdc[8306](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207920, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:52:00.787351 8326 init.cc:377] Logged in from keytab as kudu/127.8.24.62@KRBTEST.COM (short username kudu)
I20260808 16:52:00.787930 8326 webserver.cc:460] Webserver started at http://127.8.24.62:39373/ using document root <none> and password file <none>
I20260808 16:52:00.788172 8326 fs_manager.cc:359] Metadata directory not provided
I20260808 16:52:00.788220 8326 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:52:00.788383 8326 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:52:00.789778 8326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786207919384338-8288-0/minicluster-data/master-0/data/instance:
uuid: "b82e49bc9dee479d95cc28e07c71cce7"
format_stamp: "Formatted at 2026-08-08 16:52:00 on dist-test-slave-6zbq"
server_key: "01b944e5e3fd244018d8c96175dedd00"
server_key_iv: "9b60c131c9ce7d13a4654a53a9fae267"
server_key_version: "encryptionkey@0"
I20260808 16:52:00.790503 8326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786207919384338-8288-0/minicluster-data/master-0/wal/instance:
uuid: "b82e49bc9dee479d95cc28e07c71cce7"
format_stamp: "Formatted at 2026-08-08 16:52:00 on dist-test-slave-6zbq"
server_key: "01b944e5e3fd244018d8c96175dedd00"
server_key_iv: "9b60c131c9ce7d13a4654a53a9fae267"
server_key_version: "encryptionkey@0"
I20260808 16:52:00.792562 8326 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.001s
I20260808 16:52:00.793555 8348 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:00.793803 8326 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 16:52:00.793882 8326 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786207919384338-8288-0/minicluster-data/master-0/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786207919384338-8288-0/minicluster-data/master-0/wal
uuid: "b82e49bc9dee479d95cc28e07c71cce7"
format_stamp: "Formatted at 2026-08-08 16:52:00 on dist-test-slave-6zbq"
server_key: "01b944e5e3fd244018d8c96175dedd00"
server_key_iv: "9b60c131c9ce7d13a4654a53a9fae267"
server_key_version: "encryptionkey@0"
I20260808 16:52:00.793949 8326 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786207919384338-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786207919384338-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786207919384338-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:52:00.811429 8326 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:52:00.813611 8326 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:52:00.813818 8326 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:52:00.823410 8326 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.62:44065
I20260808 16:52:00.823549 8431 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.62:44065 every 8 connection(s)
I20260808 16:52:00.824086 8326 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
I20260808 16:52:00.825409 8432 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:52:00.827171 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 8326
I20260808 16:52:00.827270 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786207919384338-8288-0/minicluster-data/master-0/wal/instance
I20260808 16:52:00.827948 8288 external_mini_cluster.cc:1428] Setting key 2b936ecfc9d70e6a32f2e34b5ff4f72a
I20260808 16:52:00.828586 8432 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b82e49bc9dee479d95cc28e07c71cce7: Bootstrap starting.
I20260808 16:52:00.830171 8432 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b82e49bc9dee479d95cc28e07c71cce7: Neither blocks nor log segments found. Creating new log.
I20260808 16:52:00.831214 8432 log.cc:824] T 00000000000000000000000000000000 P b82e49bc9dee479d95cc28e07c71cce7: Log is configured to *not* fsync() on all Append() calls
I20260808 16:52:00.832270 8432 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b82e49bc9dee479d95cc28e07c71cce7: No bootstrap required, opened a new log
I20260808 16:52:00.835155 8432 raft_consensus.cc:348] T 00000000000000000000000000000000 P b82e49bc9dee479d95cc28e07c71cce7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b82e49bc9dee479d95cc28e07c71cce7" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 44065 } }
I20260808 16:52:00.835428 8432 raft_consensus.cc:374] T 00000000000000000000000000000000 P b82e49bc9dee479d95cc28e07c71cce7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:52:00.835542 8432 raft_consensus.cc:729] T 00000000000000000000000000000000 P b82e49bc9dee479d95cc28e07c71cce7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b82e49bc9dee479d95cc28e07c71cce7, State: Initialized, Role: FOLLOWER
I20260808 16:52:00.835722 8432 consensus_queue.cc:260] T 00000000000000000000000000000000 P b82e49bc9dee479d95cc28e07c71cce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b82e49bc9dee479d95cc28e07c71cce7" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 44065 } }
I20260808 16:52:00.835901 8432 raft_consensus.cc:388] T 00000000000000000000000000000000 P b82e49bc9dee479d95cc28e07c71cce7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 16:52:00.835950 8432 raft_consensus.cc:482] T 00000000000000000000000000000000 P b82e49bc9dee479d95cc28e07c71cce7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 16:52:00.836000 8432 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b82e49bc9dee479d95cc28e07c71cce7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:52:00.837105 8432 raft_consensus.cc:504] T 00000000000000000000000000000000 P b82e49bc9dee479d95cc28e07c71cce7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b82e49bc9dee479d95cc28e07c71cce7" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 44065 } }
I20260808 16:52:00.837306 8432 leader_election.cc:302] T 00000000000000000000000000000000 P b82e49bc9dee479d95cc28e07c71cce7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b82e49bc9dee479d95cc28e07c71cce7; no voters:
I20260808 16:52:00.837507 8432 leader_election.cc:288] T 00000000000000000000000000000000 P b82e49bc9dee479d95cc28e07c71cce7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 16:52:00.837541 8437 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b82e49bc9dee479d95cc28e07c71cce7 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:52:00.837706 8437 raft_consensus.cc:686] T 00000000000000000000000000000000 P b82e49bc9dee479d95cc28e07c71cce7 [term 1 LEADER]: Becoming Leader. State: Replica: b82e49bc9dee479d95cc28e07c71cce7, State: Running, Role: LEADER
I20260808 16:52:00.837898 8437 consensus_queue.cc:237] T 00000000000000000000000000000000 P b82e49bc9dee479d95cc28e07c71cce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b82e49bc9dee479d95cc28e07c71cce7" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 44065 } }
I20260808 16:52:00.838017 8432 sys_catalog.cc:564] T 00000000000000000000000000000000 P b82e49bc9dee479d95cc28e07c71cce7 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 16:52:00.838479 8440 sys_catalog.cc:455] T 00000000000000000000000000000000 P b82e49bc9dee479d95cc28e07c71cce7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b82e49bc9dee479d95cc28e07c71cce7. Latest consensus state: current_term: 1 leader_uuid: "b82e49bc9dee479d95cc28e07c71cce7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b82e49bc9dee479d95cc28e07c71cce7" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 44065 } } }
I20260808 16:52:00.838716 8440 sys_catalog.cc:458] T 00000000000000000000000000000000 P b82e49bc9dee479d95cc28e07c71cce7 [sys.catalog]: This master's current role is: LEADER
I20260808 16:52:00.838649 8438 sys_catalog.cc:455] T 00000000000000000000000000000000 P b82e49bc9dee479d95cc28e07c71cce7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b82e49bc9dee479d95cc28e07c71cce7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b82e49bc9dee479d95cc28e07c71cce7" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 44065 } } }
I20260808 16:52:00.838981 8438 sys_catalog.cc:458] T 00000000000000000000000000000000 P b82e49bc9dee479d95cc28e07c71cce7 [sys.catalog]: This master's current role is: LEADER
I20260808 16:52:00.839285 8448 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 16:52:00.840358 8448 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 16:52:00.843565 8448 catalog_manager.cc:1347] Generated new cluster ID: ca93b6f2e53e4d66845202286639f86c
I20260808 16:52:00.843621 8448 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 16:52:00 dist-test-slave-6zbq krb5kdc[8306](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207920, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260808 16:52:00.847396 8434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:00.829115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37276 (local address 127.8.24.62:44065)
0808 16:52:00.829562 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:00.829569 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:00.829988 (+ 419us) server_negotiation.cc:407] Connection header received
0808 16:52:00.834256 (+ 4268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:00.834263 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:00.834391 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:00.834501 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:00.836698 (+ 2197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:00.838284 (+ 1586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:00.840047 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:00.840725 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:00.840877 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:00.844227 (+ 3350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:00.844231 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:00.844232 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:00.844243 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:00.845642 (+ 1399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:00.846018 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:00.846020 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:00.846021 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:00.846102 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:00.846326 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:00.846327 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:00.846328 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:00.846566 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:00.846714 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:00.846978 (+ 264us) server_negotiation.cc:299] Negotiation successful
0808 16:52:00.847138 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":281,"thread_start_us":155,"threads_started":1}
I20260808 16:52:00.867244 8448 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 16:52:00.867712 8448 catalog_manager.cc:1504] Loading token signing keys...
I20260808 16:52:00.872932 8448 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b82e49bc9dee479d95cc28e07c71cce7: Generated new TSK 0
I20260808 16:52:00.873070 8448 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.24.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.1.keytab.
Entry for principal kudu/127.8.24.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.1.keytab.
I20260808 16:52:00.897679 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786207919384338-8288-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786207919384338-8288-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786207919384338-8288-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786207919384338-8288-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.1:0
--local_ip_for_outbound_sockets=127.8.24.1
--webserver_interface=127.8.24.1
--webserver_port=0
--tserver_master_addrs=127.8.24.62:44065
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.1.keytab
--principal=kudu/127.8.24.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:43829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:52:00.976456 8469 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:52:00.976626 8469 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:52:00.976647 8469 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:52:00.977753 8469 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:52:00.977794 8469 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:52:00.977818 8469 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.1
I20260808 16:52:00.979351 8469 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:43829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786207919384338-8288-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786207919384338-8288-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.1.keytab
--principal=kudu/127.8.24.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786207919384338-8288-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.24.1
--webserver_port=0
--tserver_master_addrs=127.8.24.62:44065
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8469
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.1
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786207919384338-8288-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:52:00.979638 8469 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:52:00.979856 8469 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:52:00.984728 8477 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:00.984745 8479 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:00.984851 8469 server_base.cc:1034] running on GCE node
W20260808 16:52:00.984735 8475 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:00.985101 8469 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:52:00.985314 8469 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:52:00.986438 8469 hybrid_clock.cc:648] HybridClock initialized: now 1786207920986427 us; error 30 us; skew 500 ppm
Aug 08 16:52:00 dist-test-slave-6zbq krb5kdc[8306](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207920, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:52:00.988665 8469 init.cc:377] Logged in from keytab as kudu/127.8.24.1@KRBTEST.COM (short username kudu)
I20260808 16:52:00.989039 8469 webserver.cc:460] Webserver started at http://127.8.24.1:41481/ using document root <none> and password file <none>
I20260808 16:52:00.989234 8469 fs_manager.cc:359] Metadata directory not provided
I20260808 16:52:00.989276 8469 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:52:00.989375 8469 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:52:00.990281 8469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786207919384338-8288-0/minicluster-data/ts-0/data/instance:
uuid: "21f05c92090640beba2a1680b3c6f765"
format_stamp: "Formatted at 2026-08-08 16:52:00 on dist-test-slave-6zbq"
server_key: "b888c1fdbbcc5a1397005c685ef07736"
server_key_iv: "1c0d734111f6e224937503825655a9d6"
server_key_version: "encryptionkey@0"
I20260808 16:52:00.990628 8469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786207919384338-8288-0/minicluster-data/ts-0/wal/instance:
uuid: "21f05c92090640beba2a1680b3c6f765"
format_stamp: "Formatted at 2026-08-08 16:52:00 on dist-test-slave-6zbq"
server_key: "b888c1fdbbcc5a1397005c685ef07736"
server_key_iv: "1c0d734111f6e224937503825655a9d6"
server_key_version: "encryptionkey@0"
I20260808 16:52:00.991827 8469 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 16:52:00.992532 8486 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:00.992700 8469 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 16:52:00.992780 8469 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786207919384338-8288-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786207919384338-8288-0/minicluster-data/ts-0/wal
uuid: "21f05c92090640beba2a1680b3c6f765"
format_stamp: "Formatted at 2026-08-08 16:52:00 on dist-test-slave-6zbq"
server_key: "b888c1fdbbcc5a1397005c685ef07736"
server_key_iv: "1c0d734111f6e224937503825655a9d6"
server_key_version: "encryptionkey@0"
I20260808 16:52:00.992848 8469 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786207919384338-8288-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786207919384338-8288-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786207919384338-8288-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:52:01.013782 8469 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:52:01.015708 8469 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:52:01.015828 8469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:52:01.016047 8469 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:52:01.016386 8469 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:52:01.016469 8469 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:01.016502 8469 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:52:01.016527 8469 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:01.022224 8469 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.1:46757
I20260808 16:52:01.022260 8655 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.1:46757 every 8 connection(s)
I20260808 16:52:01.022552 8469 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786207919384338-8288-0/minicluster-data/ts-0/data/info.pb
Aug 08 16:52:01 dist-test-slave-6zbq krb5kdc[8306](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207920, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.1@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260808 16:52:01.030460 8434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:01.023301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:57627 (local address 127.8.24.62:44065)
0808 16:52:01.023353 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:01.023355 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:01.023517 (+ 162us) server_negotiation.cc:407] Connection header received
0808 16:52:01.023924 (+ 407us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:01.023925 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:01.023934 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:01.023985 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:01.024946 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:01.025422 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:01.026303 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:01.026529 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:01.026578 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:01.028384 (+ 1806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:01.028385 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:01.028386 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:01.028392 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:01.029459 (+ 1067us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:01.029781 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:01.029781 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:01.029782 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:01.029825 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:01.029994 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:01.029994 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:01.029994 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:01.030079 (+ 85us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:01.030121 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:01.030306 (+ 185us) server_negotiation.cc:299] Negotiation successful
0808 16:52:01.030396 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 16:52:01.030608 8659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:01.023265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:44065 (local address 127.8.24.1:57627)
0808 16:52:01.023408 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:01.023445 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:01.023855 (+ 410us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:01.024032 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:01.024035 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:01.024707 (+ 672us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:01.024878 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:01.024882 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:01.025490 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:01.025491 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:01.026238 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:01.026240 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:01.026586 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:01.026587 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:01.026683 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:01.027071 (+ 388us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:01.027079 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:01.028306 (+ 1227us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:52:01.029521 (+ 1215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:01.029526 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:01.029528 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:01.029720 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:01.029867 (+ 147us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:01.029867 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:01.029867 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:01.029954 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:01.030119 (+ 165us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:52:01.030122 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:52:01.030221 (+ 99us) client_negotiation.cc:769] Sending connection context
0808 16:52:01.030290 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 16:52:01.030429 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":80,"threads_started":1}
I20260808 16:52:01.031078 8656 heartbeater.cc:347] Connected to a master server at 127.8.24.62:44065
I20260808 16:52:01.031159 8656 heartbeater.cc:464] Registering TS with master...
I20260808 16:52:01.031325 8656 heartbeater.cc:511] Master 127.8.24.62:44065 requested a full tablet report, sending...
I20260808 16:52:01.031905 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 8469
I20260808 16:52:01.031991 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786207919384338-8288-0/minicluster-data/ts-0/wal/instance
I20260808 16:52:01.032132 8288 external_mini_cluster.cc:1428] Setting key 92a2ebd791e67039bd2a764274da5d1c
I20260808 16:52:01.032155 8371 ts_manager.cc:194] Registered new tserver with Master: 21f05c92090640beba2a1680b3c6f765 (127.8.24.1:46757)
I20260808 16:52:01.033205 8371 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.1@KRBTEST.COM'} at 127.8.24.1:57627
WARNING: no policy specified for kudu/127.8.24.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.2.keytab.
Entry for principal kudu/127.8.24.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.2.keytab.
I20260808 16:52:01.053962 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786207919384338-8288-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786207919384338-8288-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786207919384338-8288-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786207919384338-8288-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.2:0
--local_ip_for_outbound_sockets=127.8.24.2
--webserver_interface=127.8.24.2
--webserver_port=0
--tserver_master_addrs=127.8.24.62:44065
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.2.keytab
--principal=kudu/127.8.24.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:43829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:52:01.131044 8662 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:52:01.131217 8662 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:52:01.131246 8662 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:52:01.132320 8662 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:52:01.132359 8662 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:52:01.132393 8662 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.2
I20260808 16:52:01.133769 8662 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:43829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786207919384338-8288-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786207919384338-8288-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.2.keytab
--principal=kudu/127.8.24.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786207919384338-8288-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.24.2
--webserver_port=0
--tserver_master_addrs=127.8.24.62:44065
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8662
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.2
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786207919384338-8288-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:52:01.133993 8662 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:52:01.134186 8662 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:52:01.138667 8668 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:01.138728 8671 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:01.138672 8667 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:01.138818 8662 server_base.cc:1034] running on GCE node
I20260808 16:52:01.139070 8662 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:52:01.139307 8662 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:52:01.140436 8662 hybrid_clock.cc:648] HybridClock initialized: now 1786207921140424 us; error 30 us; skew 500 ppm
Aug 08 16:52:01 dist-test-slave-6zbq krb5kdc[8306](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207921, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:52:01.142729 8662 init.cc:377] Logged in from keytab as kudu/127.8.24.2@KRBTEST.COM (short username kudu)
I20260808 16:52:01.143074 8662 webserver.cc:460] Webserver started at http://127.8.24.2:33911/ using document root <none> and password file <none>
I20260808 16:52:01.143236 8662 fs_manager.cc:359] Metadata directory not provided
I20260808 16:52:01.143275 8662 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:52:01.143359 8662 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:52:01.144254 8662 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786207919384338-8288-0/minicluster-data/ts-1/data/instance:
uuid: "33832f16eb58429ba118a8f49dfed6cb"
format_stamp: "Formatted at 2026-08-08 16:52:01 on dist-test-slave-6zbq"
server_key: "c542f412d386cfc65903015e3750dc87"
server_key_iv: "740425b85bd2500ff2a5427023e15075"
server_key_version: "encryptionkey@0"
I20260808 16:52:01.144534 8662 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786207919384338-8288-0/minicluster-data/ts-1/wal/instance:
uuid: "33832f16eb58429ba118a8f49dfed6cb"
format_stamp: "Formatted at 2026-08-08 16:52:01 on dist-test-slave-6zbq"
server_key: "c542f412d386cfc65903015e3750dc87"
server_key_iv: "740425b85bd2500ff2a5427023e15075"
server_key_version: "encryptionkey@0"
I20260808 16:52:01.145625 8662 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 16:52:01.146201 8678 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:01.146358 8662 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:01.146435 8662 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786207919384338-8288-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786207919384338-8288-0/minicluster-data/ts-1/wal
uuid: "33832f16eb58429ba118a8f49dfed6cb"
format_stamp: "Formatted at 2026-08-08 16:52:01 on dist-test-slave-6zbq"
server_key: "c542f412d386cfc65903015e3750dc87"
server_key_iv: "740425b85bd2500ff2a5427023e15075"
server_key_version: "encryptionkey@0"
I20260808 16:52:01.146485 8662 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786207919384338-8288-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786207919384338-8288-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786207919384338-8288-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:52:01.159996 8662 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:52:01.161828 8662 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:52:01.161926 8662 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:52:01.162084 8662 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:52:01.162307 8662 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:52:01.162380 8662 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:01.162407 8662 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:52:01.162422 8662 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:01.167697 8662 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.2:38315
I20260808 16:52:01.167760 8851 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.2:38315 every 8 connection(s)
I20260808 16:52:01.167989 8662 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786207919384338-8288-0/minicluster-data/ts-1/data/info.pb
Aug 08 16:52:01 dist-test-slave-6zbq krb5kdc[8306](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207921, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.2@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260808 16:52:01.175488 8434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:01.168684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:52451 (local address 127.8.24.62:44065)
0808 16:52:01.168744 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:01.168746 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:01.168886 (+ 140us) server_negotiation.cc:407] Connection header received
0808 16:52:01.169278 (+ 392us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:01.169279 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:01.169288 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:01.169335 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:01.170095 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:01.170635 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:01.171526 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:01.171700 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:01.171740 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:01.173477 (+ 1737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:01.173478 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:01.173479 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:01.173485 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:01.174560 (+ 1075us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:01.174863 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:01.174864 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:01.174865 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:01.174907 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:01.175065 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:01.175065 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:01.175066 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:01.175153 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:01.175194 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:01.175374 (+ 180us) server_negotiation.cc:299] Negotiation successful
0808 16:52:01.175428 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 16:52:01.175631 8855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:01.168655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:44065 (local address 127.8.24.2:52451)
0808 16:52:01.168787 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:01.168817 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:01.169208 (+ 391us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:01.169381 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:01.169383 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:01.169869 (+ 486us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:01.170033 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:01.170037 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:01.170704 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:01.170705 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:01.171469 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:01.171471 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:01.171747 (+ 276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:01.171748 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:01.171830 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:01.172227 (+ 397us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:01.172235 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:01.173401 (+ 1166us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:52:01.174616 (+ 1215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:01.174621 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:01.174622 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:01.174808 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:01.174947 (+ 139us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:01.174947 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:01.174948 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:01.175026 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:01.175194 (+ 168us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:52:01.175197 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:52:01.175288 (+ 91us) client_negotiation.cc:769] Sending connection context
0808 16:52:01.175359 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 16:52:01.175458 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":76,"threads_started":1}
I20260808 16:52:01.176041 8852 heartbeater.cc:347] Connected to a master server at 127.8.24.62:44065
I20260808 16:52:01.176129 8852 heartbeater.cc:464] Registering TS with master...
I20260808 16:52:01.176283 8852 heartbeater.cc:511] Master 127.8.24.62:44065 requested a full tablet report, sending...
I20260808 16:52:01.176606 8371 ts_manager.cc:194] Registered new tserver with Master: 33832f16eb58429ba118a8f49dfed6cb (127.8.24.2:38315)
I20260808 16:52:01.176924 8371 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.2@KRBTEST.COM'} at 127.8.24.2:52451
I20260808 16:52:01.177753 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 8662
I20260808 16:52:01.177832 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786207919384338-8288-0/minicluster-data/ts-1/wal/instance
I20260808 16:52:01.177973 8288 external_mini_cluster.cc:1428] Setting key ef68de38f9ace5ec73292b741d7af6ad
WARNING: no policy specified for kudu/127.8.24.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.3.keytab.
Entry for principal kudu/127.8.24.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.3.keytab.
I20260808 16:52:01.200763 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786207919384338-8288-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786207919384338-8288-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786207919384338-8288-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786207919384338-8288-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.3:0
--local_ip_for_outbound_sockets=127.8.24.3
--webserver_interface=127.8.24.3
--webserver_port=0
--tserver_master_addrs=127.8.24.62:44065
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.3.keytab
--principal=kudu/127.8.24.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:43829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:52:01.278677 8858 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:52:01.278851 8858 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:52:01.278872 8858 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:52:01.279951 8858 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:52:01.279990 8858 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:52:01.280018 8858 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.3
I20260808 16:52:01.281409 8858 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:43829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786207919384338-8288-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786207919384338-8288-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.3.keytab
--principal=kudu/127.8.24.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786207919384338-8288-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.24.3
--webserver_port=0
--tserver_master_addrs=127.8.24.62:44065
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8858
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.3
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786207919384338-8288-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:52:01.281610 8858 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:52:01.281800 8858 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:52:01.286242 8871 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:01.286253 8868 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:01.286479 8858 server_base.cc:1034] running on GCE node
W20260808 16:52:01.286260 8869 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:01.286770 8858 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:52:01.286979 8858 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:52:01.288100 8858 hybrid_clock.cc:648] HybridClock initialized: now 1786207921288086 us; error 32 us; skew 500 ppm
Aug 08 16:52:01 dist-test-slave-6zbq krb5kdc[8306](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207921, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:52:01.290288 8858 init.cc:377] Logged in from keytab as kudu/127.8.24.3@KRBTEST.COM (short username kudu)
I20260808 16:52:01.290675 8858 webserver.cc:460] Webserver started at http://127.8.24.3:41039/ using document root <none> and password file <none>
I20260808 16:52:01.290853 8858 fs_manager.cc:359] Metadata directory not provided
I20260808 16:52:01.290891 8858 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:52:01.291002 8858 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:52:01.291903 8858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786207919384338-8288-0/minicluster-data/ts-2/data/instance:
uuid: "0a47190af067436f8c05901dd5f9c903"
format_stamp: "Formatted at 2026-08-08 16:52:01 on dist-test-slave-6zbq"
server_key: "e34165d728abb526340683e9e711399f"
server_key_iv: "3951b740c042dbc6c7cca2cb76a81334"
server_key_version: "encryptionkey@0"
I20260808 16:52:01.292207 8858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786207919384338-8288-0/minicluster-data/ts-2/wal/instance:
uuid: "0a47190af067436f8c05901dd5f9c903"
format_stamp: "Formatted at 2026-08-08 16:52:01 on dist-test-slave-6zbq"
server_key: "e34165d728abb526340683e9e711399f"
server_key_iv: "3951b740c042dbc6c7cca2cb76a81334"
server_key_version: "encryptionkey@0"
I20260808 16:52:01.293313 8858 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 16:52:01.293931 8879 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:01.294065 8858 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:01.294137 8858 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786207919384338-8288-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786207919384338-8288-0/minicluster-data/ts-2/wal
uuid: "0a47190af067436f8c05901dd5f9c903"
format_stamp: "Formatted at 2026-08-08 16:52:01 on dist-test-slave-6zbq"
server_key: "e34165d728abb526340683e9e711399f"
server_key_iv: "3951b740c042dbc6c7cca2cb76a81334"
server_key_version: "encryptionkey@0"
I20260808 16:52:01.294212 8858 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786207919384338-8288-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786207919384338-8288-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786207919384338-8288-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:52:01.305370 8858 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:52:01.307251 8858 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:52:01.307364 8858 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:52:01.307543 8858 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:52:01.307796 8858 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:52:01.307827 8858 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:01.307845 8858 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:52:01.307869 8858 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:01.313252 8858 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.3:46259
I20260808 16:52:01.313311 9055 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.3:46259 every 8 connection(s)
I20260808 16:52:01.313606 8858 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786207919384338-8288-0/minicluster-data/ts-2/data/info.pb
I20260808 16:52:01.314491 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 8858
I20260808 16:52:01.314576 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786207919384338-8288-0/minicluster-data/ts-2/wal/instance
I20260808 16:52:01.314724 8288 external_mini_cluster.cc:1428] Setting key c96b4ffd02819f0c1e2ca9c3cd3b13b5
Aug 08 16:52:01 dist-test-slave-6zbq krb5kdc[8306](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207921, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.3@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260808 16:52:01.321460 8434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:01.314221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:51815 (local address 127.8.24.62:44065)
0808 16:52:01.314271 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:01.314273 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:01.314440 (+ 167us) server_negotiation.cc:407] Connection header received
0808 16:52:01.314893 (+ 453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:01.314895 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:01.314908 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:01.314978 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:01.316046 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:01.316530 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:01.317400 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:01.317583 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:01.317634 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:01.319421 (+ 1787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:01.319423 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:01.319423 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:01.319429 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:01.320450 (+ 1021us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:01.320758 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:01.320759 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:01.320759 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:01.320801 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:01.320963 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:01.320963 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:01.320964 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:01.321049 (+ 85us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:01.321091 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:01.321327 (+ 236us) server_negotiation.cc:299] Negotiation successful
0808 16:52:01.321397 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 16:52:01.321518 9059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:01.314174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:44065 (local address 127.8.24.3:51815)
0808 16:52:01.314312 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:01.314365 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:01.314797 (+ 432us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:01.315046 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:01.315050 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:01.315729 (+ 679us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:01.315971 (+ 242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:01.315976 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:01.316591 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:01.316591 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:01.317338 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:01.317340 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:01.317638 (+ 298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:01.317639 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:01.317724 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:01.318130 (+ 406us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:01.318139 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:01.319348 (+ 1209us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:52:01.320509 (+ 1161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:01.320514 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:01.320516 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:01.320705 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:01.320842 (+ 137us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:01.320842 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:01.320843 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:01.320924 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:01.321088 (+ 164us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:52:01.321090 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:52:01.321197 (+ 107us) client_negotiation.cc:769] Sending connection context
0808 16:52:01.321271 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 16:52:01.321371 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":75,"threads_started":1}
I20260808 16:52:01.321926 9056 heartbeater.cc:347] Connected to a master server at 127.8.24.62:44065
I20260808 16:52:01.322002 9056 heartbeater.cc:464] Registering TS with master...
I20260808 16:52:01.322155 9056 heartbeater.cc:511] Master 127.8.24.62:44065 requested a full tablet report, sending...
I20260808 16:52:01.322508 8371 ts_manager.cc:194] Registered new tserver with Master: 0a47190af067436f8c05901dd5f9c903 (127.8.24.3:46259)
I20260808 16:52:01.322834 8371 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.3@KRBTEST.COM'} at 127.8.24.3:51815
I20260808 16:52:01.326642 8288 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 16:52:01 dist-test-slave-6zbq krb5kdc[8306](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207921, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 16:52:01 dist-test-slave-6zbq krb5kdc[8306](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207921, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.24.1@KRBTEST.COM
I20260808 16:52:01.348969 9065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:01.341939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47986 (local address 127.8.24.1:46757)
0808 16:52:01.342077 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:01.342081 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:01.342090 (+ 9us) server_negotiation.cc:407] Connection header received
0808 16:52:01.342112 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:01.342114 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:01.342126 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:01.342198 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:01.342909 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:01.343728 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:01.344437 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:01.344744 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:01.344801 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:01.346694 (+ 1893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:01.346702 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:01.346703 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:01.346710 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:01.347883 (+ 1173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:01.348215 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:01.348218 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:01.348218 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:01.348272 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:01.348445 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:01.348445 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:01.348446 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:01.348571 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:01.348709 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:01.348786 (+ 77us) server_negotiation.cc:299] Negotiation successful
0808 16:52:01.348868 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":68,"threads_started":1}
W20260808 16:52:01.349407 8527 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47986
Aug 08 16:52:01 dist-test-slave-6zbq krb5kdc[8306](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207921, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 16:52:01 dist-test-slave-6zbq krb5kdc[8306](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207921, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.24.1@KRBTEST.COM
I20260808 16:52:01.368993 9065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:01.363271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47998 (local address 127.8.24.1:46757)
0808 16:52:01.363323 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:01.363325 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:01.363374 (+ 49us) server_negotiation.cc:407] Connection header received
0808 16:52:01.363434 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:01.363435 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:01.363445 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:01.363489 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:01.363953 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:01.364400 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:01.365050 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:01.365250 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:01.365308 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:01.367089 (+ 1781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:01.367090 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:01.367091 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:01.367097 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:01.368096 (+ 999us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:01.368370 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:01.368371 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:01.368371 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:01.368417 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:01.368571 (+ 154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:01.368572 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:01.368572 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:01.368654 (+ 82us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:01.368751 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:01.368843 (+ 92us) server_negotiation.cc:299] Negotiation successful
0808 16:52:01.368911 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 16:52:01.369387 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 8469
I20260808 16:52:01.383387 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 8662
I20260808 16:52:01.400161 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 8858
I20260808 16:52:01.410526 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 8326
2026-08-08T16:52:01Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2020 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786207919384338-8288-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 16:52:01 dist-test-slave-6zbq krb5kdc[9075](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 16:52:01 dist-test-slave-6zbq krb5kdc[9075](info): set up 2 sockets
Aug 08 16:52:01 dist-test-slave-6zbq krb5kdc[9075](info): commencing operation
krb5kdc: starting...
W20260808 16:52:02.541224 8288 mini_kdc.cc:121] Time spent starting KDC: real 1.109s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786207919384338-8288-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786207919384338-8288-0/krb5kdc/test-user.keytab.
Aug 08 16:52:02 dist-test-slave-6zbq krb5kdc[9075](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207922, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T16:52:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T16:52:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.24.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
I20260808 16:52:02.646016 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786207919384338-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786207919384338-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786207919384338-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:41511
--webserver_interface=127.8.24.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:42795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:41511
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:52:02.723796 9095 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:52:02.723981 9095 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:52:02.724004 9095 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:52:02.725075 9095 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 16:52:02.725118 9095 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:52:02.725132 9095 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:52:02.725148 9095 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 16:52:02.725159 9095 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 16:52:02.726591 9095 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:42795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786207919384338-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786207919384338-8288-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:41511
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:41511
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.24.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9095
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786207919384338-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:52:02.726766 9095 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:52:02.726919 9095 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:52:02.731300 9100 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:02.731274 9103 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:02.731375 9101 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:02.731403 9095 server_base.cc:1034] running on GCE node
I20260808 16:52:02.731673 9095 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:52:02.731922 9095 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:52:02.733045 9095 hybrid_clock.cc:648] HybridClock initialized: now 1786207922733023 us; error 39 us; skew 500 ppm
Aug 08 16:52:02 dist-test-slave-6zbq krb5kdc[9075](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207922, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:52:02.735309 9095 init.cc:377] Logged in from keytab as kudu/127.8.24.62@KRBTEST.COM (short username kudu)
I20260808 16:52:02.735690 9095 webserver.cc:460] Webserver started at http://127.8.24.62:38665/ using document root <none> and password file <none>
I20260808 16:52:02.735853 9095 fs_manager.cc:359] Metadata directory not provided
I20260808 16:52:02.735890 9095 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:52:02.735976 9095 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:52:02.736894 9095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786207919384338-8288-0/minicluster-data/master-0/data/instance:
uuid: "33650475f56a44fd9eaccf9bed445ba1"
format_stamp: "Formatted at 2026-08-08 16:52:02 on dist-test-slave-6zbq"
server_key: "c1b7c4f6c7038b30df23b66916ac62e3"
server_key_iv: "d3ee4d0e75bf4c85ed39f2e681aa43e9"
server_key_version: "encryptionkey@0"
I20260808 16:52:02.737178 9095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786207919384338-8288-0/minicluster-data/master-0/wal/instance:
uuid: "33650475f56a44fd9eaccf9bed445ba1"
format_stamp: "Formatted at 2026-08-08 16:52:02 on dist-test-slave-6zbq"
server_key: "c1b7c4f6c7038b30df23b66916ac62e3"
server_key_iv: "d3ee4d0e75bf4c85ed39f2e681aa43e9"
server_key_version: "encryptionkey@0"
I20260808 16:52:02.738297 9095 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 16:52:02.738962 9113 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:02.739116 9095 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 16:52:02.739197 9095 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786207919384338-8288-0/minicluster-data/master-0/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786207919384338-8288-0/minicluster-data/master-0/wal
uuid: "33650475f56a44fd9eaccf9bed445ba1"
format_stamp: "Formatted at 2026-08-08 16:52:02 on dist-test-slave-6zbq"
server_key: "c1b7c4f6c7038b30df23b66916ac62e3"
server_key_iv: "d3ee4d0e75bf4c85ed39f2e681aa43e9"
server_key_version: "encryptionkey@0"
I20260808 16:52:02.739246 9095 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786207919384338-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786207919384338-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786207919384338-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:52:02.769337 9095 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:52:02.770917 9095 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:52:02.771015 9095 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:52:02.774461 9095 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.62:41511
I20260808 16:52:02.774502 9193 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.62:41511 every 8 connection(s)
I20260808 16:52:02.774775 9095 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
I20260808 16:52:02.775512 9194 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:52:02.777540 9194 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33650475f56a44fd9eaccf9bed445ba1: Bootstrap starting.
I20260808 16:52:02.778062 9194 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 33650475f56a44fd9eaccf9bed445ba1: Neither blocks nor log segments found. Creating new log.
I20260808 16:52:02.778329 9194 log.cc:824] T 00000000000000000000000000000000 P 33650475f56a44fd9eaccf9bed445ba1: Log is configured to *not* fsync() on all Append() calls
I20260808 16:52:02.778947 9194 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33650475f56a44fd9eaccf9bed445ba1: No bootstrap required, opened a new log
I20260808 16:52:02.779819 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 9095
I20260808 16:52:02.779883 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786207919384338-8288-0/minicluster-data/master-0/wal/instance
I20260808 16:52:02.780015 8288 external_mini_cluster.cc:1428] Setting key eb9deedced29a11af5099c433c8648c9
I20260808 16:52:02.780656 9194 raft_consensus.cc:348] T 00000000000000000000000000000000 P 33650475f56a44fd9eaccf9bed445ba1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33650475f56a44fd9eaccf9bed445ba1" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 41511 } }
I20260808 16:52:02.780789 9194 raft_consensus.cc:374] T 00000000000000000000000000000000 P 33650475f56a44fd9eaccf9bed445ba1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:52:02.780822 9194 raft_consensus.cc:729] T 00000000000000000000000000000000 P 33650475f56a44fd9eaccf9bed445ba1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33650475f56a44fd9eaccf9bed445ba1, State: Initialized, Role: FOLLOWER
I20260808 16:52:02.780942 9194 consensus_queue.cc:260] T 00000000000000000000000000000000 P 33650475f56a44fd9eaccf9bed445ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33650475f56a44fd9eaccf9bed445ba1" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 41511 } }
I20260808 16:52:02.781013 9194 raft_consensus.cc:388] T 00000000000000000000000000000000 P 33650475f56a44fd9eaccf9bed445ba1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 16:52:02.781044 9194 raft_consensus.cc:482] T 00000000000000000000000000000000 P 33650475f56a44fd9eaccf9bed445ba1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 16:52:02.781090 9194 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 33650475f56a44fd9eaccf9bed445ba1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:52:02.781955 9194 raft_consensus.cc:504] T 00000000000000000000000000000000 P 33650475f56a44fd9eaccf9bed445ba1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33650475f56a44fd9eaccf9bed445ba1" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 41511 } }
I20260808 16:52:02.782048 9194 leader_election.cc:302] T 00000000000000000000000000000000 P 33650475f56a44fd9eaccf9bed445ba1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 33650475f56a44fd9eaccf9bed445ba1; no voters:
I20260808 16:52:02.782200 9194 leader_election.cc:288] T 00000000000000000000000000000000 P 33650475f56a44fd9eaccf9bed445ba1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 16:52:02.782248 9200 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 33650475f56a44fd9eaccf9bed445ba1 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:52:02.782371 9200 raft_consensus.cc:686] T 00000000000000000000000000000000 P 33650475f56a44fd9eaccf9bed445ba1 [term 1 LEADER]: Becoming Leader. State: Replica: 33650475f56a44fd9eaccf9bed445ba1, State: Running, Role: LEADER
I20260808 16:52:02.782482 9194 sys_catalog.cc:564] T 00000000000000000000000000000000 P 33650475f56a44fd9eaccf9bed445ba1 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 16:52:02.782454 9200 consensus_queue.cc:237] T 00000000000000000000000000000000 P 33650475f56a44fd9eaccf9bed445ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33650475f56a44fd9eaccf9bed445ba1" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 41511 } }
I20260808 16:52:02.783486 9202 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33650475f56a44fd9eaccf9bed445ba1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 33650475f56a44fd9eaccf9bed445ba1. Latest consensus state: current_term: 1 leader_uuid: "33650475f56a44fd9eaccf9bed445ba1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33650475f56a44fd9eaccf9bed445ba1" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 41511 } } }
I20260808 16:52:02.783576 9202 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33650475f56a44fd9eaccf9bed445ba1 [sys.catalog]: This master's current role is: LEADER
I20260808 16:52:02.783782 9201 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33650475f56a44fd9eaccf9bed445ba1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "33650475f56a44fd9eaccf9bed445ba1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33650475f56a44fd9eaccf9bed445ba1" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 41511 } } }
I20260808 16:52:02.783969 9201 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33650475f56a44fd9eaccf9bed445ba1 [sys.catalog]: This master's current role is: LEADER
W20260808 16:52:02.784262 9223 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 33650475f56a44fd9eaccf9bed445ba1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 16:52:02.784317 9223 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 16:52:02.784529 9221 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 16:52:02.784736 9221 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 16:52:02.786134 9221 catalog_manager.cc:1347] Generated new cluster ID: ab38605f07a141ab85662503ae404868
I20260808 16:52:02.786182 9221 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 16:52:02 dist-test-slave-6zbq krb5kdc[9075](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207922, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260808 16:52:02.790144 9199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:02.780464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55576 (local address 127.8.24.62:41511)
0808 16:52:02.780618 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:02.780625 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:02.780648 (+ 23us) server_negotiation.cc:407] Connection header received
0808 16:52:02.780733 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:02.780738 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:02.780776 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:02.780892 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:02.781585 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:02.782621 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:02.783601 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:02.784169 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:02.784268 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:02.786792 (+ 2524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:02.786796 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:02.786797 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:02.786807 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:02.788615 (+ 1808us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:02.788973 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:02.788976 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:02.788977 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:02.789096 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:02.789266 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:02.789267 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:02.789267 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:02.789585 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:02.789750 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:02.789816 (+ 66us) server_negotiation.cc:299] Negotiation successful
0808 16:52:02.789946 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":74,"threads_started":1}
I20260808 16:52:02.793678 9221 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 16:52:02.794134 9221 catalog_manager.cc:1504] Loading token signing keys...
I20260808 16:52:02.796770 9221 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 33650475f56a44fd9eaccf9bed445ba1: Generated new TSK 0
I20260808 16:52:02.796913 9221 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.24.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.1.keytab.
Entry for principal kudu/127.8.24.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.1.keytab.
I20260808 16:52:02.826483 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786207919384338-8288-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786207919384338-8288-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786207919384338-8288-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786207919384338-8288-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.1:0
--local_ip_for_outbound_sockets=127.8.24.1
--webserver_interface=127.8.24.1
--webserver_port=0
--tserver_master_addrs=127.8.24.62:41511
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.1.keytab
--principal=kudu/127.8.24.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:42795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:52:02.927404 9230 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:52:02.927584 9230 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:52:02.927604 9230 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:52:02.928683 9230 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:52:02.928721 9230 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:52:02.928746 9230 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.1
I20260808 16:52:02.930145 9230 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:42795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786207919384338-8288-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786207919384338-8288-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.1.keytab
--principal=kudu/127.8.24.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786207919384338-8288-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.24.1
--webserver_port=0
--tserver_master_addrs=127.8.24.62:41511
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9230
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.1
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786207919384338-8288-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:52:02.930370 9230 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:52:02.930573 9230 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:52:02.934980 9240 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:02.934985 9237 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:02.934984 9238 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:02.935122 9230 server_base.cc:1034] running on GCE node
I20260808 16:52:02.935393 9230 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:52:02.935576 9230 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:52:02.936702 9230 hybrid_clock.cc:648] HybridClock initialized: now 1786207922936687 us; error 29 us; skew 500 ppm
Aug 08 16:52:02 dist-test-slave-6zbq krb5kdc[9075](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207922, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:52:02.938925 9230 init.cc:377] Logged in from keytab as kudu/127.8.24.1@KRBTEST.COM (short username kudu)
I20260808 16:52:02.939299 9230 webserver.cc:460] Webserver started at http://127.8.24.1:39561/ using document root <none> and password file <none>
I20260808 16:52:02.939479 9230 fs_manager.cc:359] Metadata directory not provided
I20260808 16:52:02.939523 9230 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:52:02.939637 9230 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:52:02.940557 9230 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786207919384338-8288-0/minicluster-data/ts-0/data/instance:
uuid: "4f4d7ace52364ea0915721a36afc6e37"
format_stamp: "Formatted at 2026-08-08 16:52:02 on dist-test-slave-6zbq"
server_key: "a2aabe603d8eec3abd26b6210ec100d1"
server_key_iv: "3bcb3dd08019a2e23d1b57f4e1551fbe"
server_key_version: "encryptionkey@0"
I20260808 16:52:02.940886 9230 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786207919384338-8288-0/minicluster-data/ts-0/wal/instance:
uuid: "4f4d7ace52364ea0915721a36afc6e37"
format_stamp: "Formatted at 2026-08-08 16:52:02 on dist-test-slave-6zbq"
server_key: "a2aabe603d8eec3abd26b6210ec100d1"
server_key_iv: "3bcb3dd08019a2e23d1b57f4e1551fbe"
server_key_version: "encryptionkey@0"
I20260808 16:52:02.942154 9230 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 16:52:02.942824 9247 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:02.942952 9230 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:02.943032 9230 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786207919384338-8288-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786207919384338-8288-0/minicluster-data/ts-0/wal
uuid: "4f4d7ace52364ea0915721a36afc6e37"
format_stamp: "Formatted at 2026-08-08 16:52:02 on dist-test-slave-6zbq"
server_key: "a2aabe603d8eec3abd26b6210ec100d1"
server_key_iv: "3bcb3dd08019a2e23d1b57f4e1551fbe"
server_key_version: "encryptionkey@0"
I20260808 16:52:02.943089 9230 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786207919384338-8288-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786207919384338-8288-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786207919384338-8288-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:52:02.975092 9230 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:52:02.976595 9230 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:52:02.976713 9230 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:52:02.976895 9230 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:52:02.977145 9230 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:52:02.977176 9230 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:02.977202 9230 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:52:02.977226 9230 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:02.982421 9230 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.1:35155
I20260808 16:52:02.982479 9418 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.1:35155 every 8 connection(s)
I20260808 16:52:02.982723 9230 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786207919384338-8288-0/minicluster-data/ts-0/data/info.pb
Aug 08 16:52:02 dist-test-slave-6zbq krb5kdc[9075](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207922, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.1@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260808 16:52:02.990695 9199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:02.983323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:40061 (local address 127.8.24.62:41511)
0808 16:52:02.983382 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:02.983385 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:02.983599 (+ 214us) server_negotiation.cc:407] Connection header received
0808 16:52:02.984697 (+ 1098us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:02.984698 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:02.984709 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:02.984755 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:02.985261 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:02.985738 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:02.986682 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:02.986869 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:02.986909 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:02.988633 (+ 1724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:02.988634 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:02.988634 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:02.988642 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:02.989689 (+ 1047us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:02.990004 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:02.990005 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:02.990005 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:02.990045 (+ 40us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:02.990209 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:02.990209 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:02.990210 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:02.990305 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:02.990374 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:02.990575 (+ 201us) server_negotiation.cc:299] Negotiation successful
0808 16:52:02.990634 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 16:52:02.990849 9423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:02.983317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:41511 (local address 127.8.24.1:40061)
0808 16:52:02.983484 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:02.983516 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:02.984613 (+ 1097us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:02.984810 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:02.984813 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:02.985033 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:02.985198 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:02.985202 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:02.985795 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:02.985796 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:02.986620 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:02.986622 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:02.986917 (+ 295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:02.986918 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:02.987002 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:02.987354 (+ 352us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:02.987361 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:02.988554 (+ 1193us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:52:02.989754 (+ 1200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:02.989759 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:02.989760 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:02.989951 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:02.990087 (+ 136us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:02.990088 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:02.990088 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:02.990169 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:02.990368 (+ 199us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:52:02.990373 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:52:02.990479 (+ 106us) client_negotiation.cc:769] Sending connection context
0808 16:52:02.990556 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 16:52:02.990661 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":99,"threads_started":1}
I20260808 16:52:02.991288 9420 heartbeater.cc:347] Connected to a master server at 127.8.24.62:41511
I20260808 16:52:02.991364 9420 heartbeater.cc:464] Registering TS with master...
I20260808 16:52:02.991415 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 9230
I20260808 16:52:02.991504 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786207919384338-8288-0/minicluster-data/ts-0/wal/instance
I20260808 16:52:02.991523 9420 heartbeater.cc:511] Master 127.8.24.62:41511 requested a full tablet report, sending...
I20260808 16:52:02.991652 8288 external_mini_cluster.cc:1428] Setting key 8880944a17a4c610970c9c0b24eb2afb
I20260808 16:52:02.992019 9137 ts_manager.cc:194] Registered new tserver with Master: 4f4d7ace52364ea0915721a36afc6e37 (127.8.24.1:35155)
I20260808 16:52:02.992748 9137 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.1@KRBTEST.COM'} at 127.8.24.1:40061
WARNING: no policy specified for kudu/127.8.24.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.2.keytab.
Entry for principal kudu/127.8.24.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.2.keytab.
I20260808 16:52:03.013921 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786207919384338-8288-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786207919384338-8288-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786207919384338-8288-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786207919384338-8288-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.2:0
--local_ip_for_outbound_sockets=127.8.24.2
--webserver_interface=127.8.24.2
--webserver_port=0
--tserver_master_addrs=127.8.24.62:41511
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.2.keytab
--principal=kudu/127.8.24.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:42795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:52:03.090507 9426 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:52:03.090667 9426 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:52:03.090698 9426 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:52:03.091820 9426 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:52:03.091858 9426 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:52:03.091894 9426 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.2
I20260808 16:52:03.093293 9426 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:42795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786207919384338-8288-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786207919384338-8288-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.2.keytab
--principal=kudu/127.8.24.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786207919384338-8288-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.24.2
--webserver_port=0
--tserver_master_addrs=127.8.24.62:41511
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9426
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.2
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786207919384338-8288-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:52:03.093494 9426 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:52:03.093683 9426 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:52:03.097891 9435 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:03.097965 9433 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:03.097982 9437 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:03.098132 9426 server_base.cc:1034] running on GCE node
I20260808 16:52:03.098320 9426 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:52:03.098582 9426 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:52:03.099714 9426 hybrid_clock.cc:648] HybridClock initialized: now 1786207923099697 us; error 28 us; skew 500 ppm
Aug 08 16:52:03 dist-test-slave-6zbq krb5kdc[9075](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207923, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:52:03.101831 9426 init.cc:377] Logged in from keytab as kudu/127.8.24.2@KRBTEST.COM (short username kudu)
I20260808 16:52:03.102197 9426 webserver.cc:460] Webserver started at http://127.8.24.2:40433/ using document root <none> and password file <none>
I20260808 16:52:03.102428 9426 fs_manager.cc:359] Metadata directory not provided
I20260808 16:52:03.102476 9426 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:52:03.102591 9426 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:52:03.103487 9426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786207919384338-8288-0/minicluster-data/ts-1/data/instance:
uuid: "fb0f7086a3144f70abc4fdc93c1be0c3"
format_stamp: "Formatted at 2026-08-08 16:52:03 on dist-test-slave-6zbq"
server_key: "c864d654110f91c0bb55bb06c6ad010e"
server_key_iv: "3b368c8eda97348fafedf20be3597fe1"
server_key_version: "encryptionkey@0"
I20260808 16:52:03.103786 9426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786207919384338-8288-0/minicluster-data/ts-1/wal/instance:
uuid: "fb0f7086a3144f70abc4fdc93c1be0c3"
format_stamp: "Formatted at 2026-08-08 16:52:03 on dist-test-slave-6zbq"
server_key: "c864d654110f91c0bb55bb06c6ad010e"
server_key_iv: "3b368c8eda97348fafedf20be3597fe1"
server_key_version: "encryptionkey@0"
I20260808 16:52:03.104904 9426 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 16:52:03.105468 9451 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:03.105610 9426 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 16:52:03.105679 9426 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786207919384338-8288-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786207919384338-8288-0/minicluster-data/ts-1/wal
uuid: "fb0f7086a3144f70abc4fdc93c1be0c3"
format_stamp: "Formatted at 2026-08-08 16:52:03 on dist-test-slave-6zbq"
server_key: "c864d654110f91c0bb55bb06c6ad010e"
server_key_iv: "3b368c8eda97348fafedf20be3597fe1"
server_key_version: "encryptionkey@0"
I20260808 16:52:03.105739 9426 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786207919384338-8288-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786207919384338-8288-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786207919384338-8288-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:52:03.119128 9426 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:52:03.120579 9426 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:52:03.120698 9426 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:52:03.120885 9426 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:52:03.121148 9426 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:52:03.121181 9426 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:03.121204 9426 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:52:03.121222 9426 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:03.126384 9426 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.2:39573
I20260808 16:52:03.126461 9620 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.2:39573 every 8 connection(s)
I20260808 16:52:03.126672 9426 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786207919384338-8288-0/minicluster-data/ts-1/data/info.pb
I20260808 16:52:03.127581 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 9426
I20260808 16:52:03.127671 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786207919384338-8288-0/minicluster-data/ts-1/wal/instance
I20260808 16:52:03.127838 8288 external_mini_cluster.cc:1428] Setting key e24efc7e3b25bbea917f912cec872b24
Aug 08 16:52:03 dist-test-slave-6zbq krb5kdc[9075](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207923, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.2@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260808 16:52:03.135653 9199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:03.127222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:47455 (local address 127.8.24.62:41511)
0808 16:52:03.127276 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:03.127278 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:03.127459 (+ 181us) server_negotiation.cc:407] Connection header received
0808 16:52:03.128455 (+ 996us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:03.128455 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:03.128465 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:03.128510 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:03.129042 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:03.129525 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:03.130866 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:03.131127 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:03.131201 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:03.133151 (+ 1950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:03.133152 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:03.133153 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:03.133159 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:03.134405 (+ 1246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:03.134828 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:03.134828 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:03.134829 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:03.134873 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:03.135109 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:03.135109 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:03.135110 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:03.135198 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:03.135239 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:03.135524 (+ 285us) server_negotiation.cc:299] Negotiation successful
0808 16:52:03.135592 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 16:52:03.135880 9627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:03.127204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:41511 (local address 127.8.24.2:47455)
0808 16:52:03.127340 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:03.127369 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:03.128370 (+ 1001us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:03.128581 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:03.128584 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:03.128800 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:03.128959 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:03.128963 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:03.129602 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:03.129603 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:03.130788 (+ 1185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:03.130792 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:03.131202 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:03.131202 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:03.131304 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:03.131708 (+ 404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:03.131715 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:03.133065 (+ 1350us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:52:03.134471 (+ 1406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:03.134478 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:03.134480 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:03.134759 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:03.134922 (+ 163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:03.134923 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:03.134923 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:03.135052 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:03.135249 (+ 197us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:52:03.135254 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:52:03.135400 (+ 146us) client_negotiation.cc:769] Sending connection context
0808 16:52:03.135516 (+ 116us) client_negotiation.cc:240] Negotiation successful
0808 16:52:03.135672 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":81,"threads_started":1}
WARNING: no policy specified for kudu/127.8.24.3@KRBTEST.COM; defaulting to no policy
I20260808 16:52:03.136328 9622 heartbeater.cc:347] Connected to a master server at 127.8.24.62:41511
I20260808 16:52:03.136416 9622 heartbeater.cc:464] Registering TS with master...
I20260808 16:52:03.136577 9622 heartbeater.cc:511] Master 127.8.24.62:41511 requested a full tablet report, sending...
I20260808 16:52:03.136866 9137 ts_manager.cc:194] Registered new tserver with Master: fb0f7086a3144f70abc4fdc93c1be0c3 (127.8.24.2:39573)
I20260808 16:52:03.137231 9137 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.2@KRBTEST.COM'} at 127.8.24.2:47455
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.3.keytab.
Entry for principal kudu/127.8.24.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.3.keytab.
I20260808 16:52:03.150611 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786207919384338-8288-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786207919384338-8288-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786207919384338-8288-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786207919384338-8288-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.3:0
--local_ip_for_outbound_sockets=127.8.24.3
--webserver_interface=127.8.24.3
--webserver_port=0
--tserver_master_addrs=127.8.24.62:41511
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.3.keytab
--principal=kudu/127.8.24.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:42795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:52:03.226857 9631 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:52:03.227023 9631 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:52:03.227044 9631 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:52:03.228093 9631 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:52:03.228127 9631 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:52:03.228152 9631 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.3
I20260808 16:52:03.229524 9631 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:42795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786207919384338-8288-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786207919384338-8288-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.3.keytab
--principal=kudu/127.8.24.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786207919384338-8288-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.24.3
--webserver_port=0
--tserver_master_addrs=127.8.24.62:41511
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9631
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.3
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786207919384338-8288-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:52:03.229684 9631 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:52:03.229847 9631 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:52:03.234203 9638 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:03.234277 9631 server_base.cc:1034] running on GCE node
W20260808 16:52:03.234193 9640 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:03.234226 9644 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:03.234544 9631 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:52:03.234759 9631 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:52:03.235893 9631 hybrid_clock.cc:648] HybridClock initialized: now 1786207923235880 us; error 28 us; skew 500 ppm
Aug 08 16:52:03 dist-test-slave-6zbq krb5kdc[9075](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207923, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:52:03.238009 9631 init.cc:377] Logged in from keytab as kudu/127.8.24.3@KRBTEST.COM (short username kudu)
I20260808 16:52:03.238433 9631 webserver.cc:460] Webserver started at http://127.8.24.3:36407/ using document root <none> and password file <none>
I20260808 16:52:03.238641 9631 fs_manager.cc:359] Metadata directory not provided
I20260808 16:52:03.238682 9631 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:52:03.238811 9631 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:52:03.239728 9631 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786207919384338-8288-0/minicluster-data/ts-2/data/instance:
uuid: "b06bd78a89f042ce8df08e88f3581014"
format_stamp: "Formatted at 2026-08-08 16:52:03 on dist-test-slave-6zbq"
server_key: "d5c61e2fc4b700f75b2bd6db3c591e29"
server_key_iv: "631520ce24f708ec9cd32b35661a111c"
server_key_version: "encryptionkey@0"
I20260808 16:52:03.240041 9631 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786207919384338-8288-0/minicluster-data/ts-2/wal/instance:
uuid: "b06bd78a89f042ce8df08e88f3581014"
format_stamp: "Formatted at 2026-08-08 16:52:03 on dist-test-slave-6zbq"
server_key: "d5c61e2fc4b700f75b2bd6db3c591e29"
server_key_iv: "631520ce24f708ec9cd32b35661a111c"
server_key_version: "encryptionkey@0"
I20260808 16:52:03.241274 9631 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 16:52:03.241946 9652 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:03.242089 9631 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 16:52:03.242172 9631 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786207919384338-8288-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786207919384338-8288-0/minicluster-data/ts-2/wal
uuid: "b06bd78a89f042ce8df08e88f3581014"
format_stamp: "Formatted at 2026-08-08 16:52:03 on dist-test-slave-6zbq"
server_key: "d5c61e2fc4b700f75b2bd6db3c591e29"
server_key_iv: "631520ce24f708ec9cd32b35661a111c"
server_key_version: "encryptionkey@0"
I20260808 16:52:03.242256 9631 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786207919384338-8288-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786207919384338-8288-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786207919384338-8288-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:52:03.253026 9631 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:52:03.254662 9631 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:52:03.254781 9631 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:52:03.256289 9631 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:52:03.256551 9631 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:52:03.256582 9631 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:03.256608 9631 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:52:03.256632 9631 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:03.261958 9631 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.3:36327
I20260808 16:52:03.262012 9819 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.3:36327 every 8 connection(s)
I20260808 16:52:03.262264 9631 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786207919384338-8288-0/minicluster-data/ts-2/data/info.pb
I20260808 16:52:03.264258 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 9631
I20260808 16:52:03.264340 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786207919384338-8288-0/minicluster-data/ts-2/wal/instance
I20260808 16:52:03.264472 8288 external_mini_cluster.cc:1428] Setting key ffec3405ee9d2add7101fcf116733403
Aug 08 16:52:03 dist-test-slave-6zbq krb5kdc[9075](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207923, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.3@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260808 16:52:03.270282 9199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:03.262882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:42235 (local address 127.8.24.62:41511)
0808 16:52:03.262930 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:03.262932 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:03.263104 (+ 172us) server_negotiation.cc:407] Connection header received
0808 16:52:03.263959 (+ 855us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:03.263960 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:03.263974 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:03.264021 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:03.264670 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:03.265352 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:03.266259 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:03.266484 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:03.266526 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:03.268297 (+ 1771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:03.268299 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:03.268299 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:03.268306 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:03.269318 (+ 1012us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:03.269623 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:03.269623 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:03.269624 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:03.269664 (+ 40us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:03.269839 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:03.269840 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:03.269840 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:03.269928 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:03.269969 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:03.270168 (+ 199us) server_negotiation.cc:299] Negotiation successful
0808 16:52:03.270220 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 16:52:03.270478 9822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:03.262839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:41511 (local address 127.8.24.3:42235)
0808 16:52:03.262977 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:03.263018 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:03.263851 (+ 833us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:03.264095 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:03.264098 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:03.264364 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:03.264582 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:03.264588 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:03.265424 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:03.265425 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:03.266193 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:03.266195 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:03.266544 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:03.266545 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:03.266640 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:03.267040 (+ 400us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:03.267048 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:03.268227 (+ 1179us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:52:03.269375 (+ 1148us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:03.269380 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:03.269382 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:03.269570 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:03.269706 (+ 136us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:03.269707 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:03.269707 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:03.269788 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:03.269972 (+ 184us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:52:03.269977 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:52:03.270076 (+ 99us) client_negotiation.cc:769] Sending connection context
0808 16:52:03.270151 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 16:52:03.270255 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":68,"threads_started":1}
I20260808 16:52:03.270836 9820 heartbeater.cc:347] Connected to a master server at 127.8.24.62:41511
I20260808 16:52:03.270910 9820 heartbeater.cc:464] Registering TS with master...
I20260808 16:52:03.271068 9820 heartbeater.cc:511] Master 127.8.24.62:41511 requested a full tablet report, sending...
I20260808 16:52:03.271409 9137 ts_manager.cc:194] Registered new tserver with Master: b06bd78a89f042ce8df08e88f3581014 (127.8.24.3:36327)
I20260808 16:52:03.271744 9137 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.3@KRBTEST.COM'} at 127.8.24.3:42235
I20260808 16:52:03.275898 8288 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 16:52:03.282579 9199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:03.277618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55578 (local address 127.8.24.62:41511)
0808 16:52:03.277676 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:03.277678 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:03.277772 (+ 94us) server_negotiation.cc:407] Connection header received
0808 16:52:03.277823 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:03.277824 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:03.277832 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:03.277876 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:03.278395 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:03.278831 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:03.279500 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:03.279683 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:03.279734 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:03.280654 (+ 920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:03.280655 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:03.280655 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:03.280662 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:03.281575 (+ 913us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:03.281880 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:03.281881 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:03.281882 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:03.281926 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:03.282082 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:03.282083 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:03.282083 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:03.282209 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:03.282293 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:03.282435 (+ 142us) server_negotiation.cc:299] Negotiation successful
0808 16:52:03.282500 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 16:52:03.284987 9137 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55578:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 16:52:03.286691 9137 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 16:52:03.292764 9830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:03.289182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.3:36327 (local address 127.0.0.1:52910)
0808 16:52:03.289471 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:03.289480 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:03.289545 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:03.289698 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:03.289701 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:03.289703 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:52:03.289759 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:03.289761 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:03.290982 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:03.290982 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:03.291797 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:03.291799 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:03.292485 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:03.292485 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:03.292574 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:03.292587 (+ 13us) client_negotiation.cc:769] Sending connection context
0808 16:52:03.292652 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 16:52:03.292696 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":48,"threads_started":1}
I20260808 16:52:03.292769 9832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:03.289276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52910 (local address 127.8.24.3:36327)
0808 16:52:03.289426 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:03.289430 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:03.289559 (+ 129us) server_negotiation.cc:407] Connection header received
0808 16:52:03.289600 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:03.289603 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:03.289614 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:03.289675 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:52:03.289856 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:03.290872 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:03.291873 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:03.292421 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:03.292486 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:03.292535 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:03.292670 (+ 135us) server_negotiation.cc:299] Negotiation successful
0808 16:52:03.292704 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":71,"threads_started":1}
I20260808 16:52:03.293354 9835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:03.289451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44888 (local address 127.8.24.1:35155)
0808 16:52:03.289803 (+ 352us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:03.289807 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:03.289817 (+ 10us) server_negotiation.cc:407] Connection header received
0808 16:52:03.289852 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:03.289854 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:03.289869 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:03.289937 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:52:03.290126 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:03.291145 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:03.292352 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:03.293051 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:03.293129 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:03.293183 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:03.293266 (+ 83us) server_negotiation.cc:299] Negotiation successful
0808 16:52:03.293297 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":74,"threads_started":1}
I20260808 16:52:03.293365 9833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:03.289329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:35155 (local address 127.0.0.1:44888)
0808 16:52:03.289608 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:03.289615 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:03.289647 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:03.290017 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:03.290018 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:03.290019 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:52:03.290054 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:03.290055 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:03.291235 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:03.291236 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:03.292272 (+ 1036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:03.292274 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:03.293133 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:03.293134 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:03.293194 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:03.293198 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 16:52:03.293252 (+ 54us) client_negotiation.cc:240] Negotiation successful
0808 16:52:03.293297 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":48,"threads_started":1}
I20260808 16:52:03.293653 9717 tablet_service.cc:1467] Processing CreateTablet for tablet 60b9c6e097d0455a9c2263069fa66563 (DEFAULT_TABLE table=test-table [id=48a8c27b0f954b439124e61b18e450a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:52:03.293977 9717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 60b9c6e097d0455a9c2263069fa66563. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:52:03.294008 9323 tablet_service.cc:1467] Processing CreateTablet for tablet 60b9c6e097d0455a9c2263069fa66563 (DEFAULT_TABLE table=test-table [id=48a8c27b0f954b439124e61b18e450a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:52:03.294329 9323 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 60b9c6e097d0455a9c2263069fa66563. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:52:03.294620 9829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:03.289133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.2:39573 (local address 127.0.0.1:57466)
0808 16:52:03.289406 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:03.289441 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:03.289494 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:03.289802 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:03.289803 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:03.289805 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:52:03.289857 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:03.289858 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:03.290897 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:03.290898 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:03.292077 (+ 1179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:03.292080 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:03.293171 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:03.293172 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:03.294455 (+ 1283us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:03.294460 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 16:52:03.294517 (+ 57us) client_negotiation.cc:240] Negotiation successful
0808 16:52:03.294555 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"mutex_wait_us":35,"thread_start_us":54,"threads_started":1}
I20260808 16:52:03.294910 9831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:03.289232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57466 (local address 127.8.24.2:39573)
0808 16:52:03.289653 (+ 421us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:03.289658 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:03.289668 (+ 10us) server_negotiation.cc:407] Connection header received
0808 16:52:03.289704 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:03.289706 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:03.289720 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:03.289798 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:52:03.289916 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:03.290813 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:03.292175 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:03.293029 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:03.293087 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:03.293142 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:03.294795 (+ 1653us) server_negotiation.cc:299] Negotiation successful
0808 16:52:03.294840 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382,"thread_start_us":81,"threads_started":1}
I20260808 16:52:03.295415 9520 tablet_service.cc:1467] Processing CreateTablet for tablet 60b9c6e097d0455a9c2263069fa66563 (DEFAULT_TABLE table=test-table [id=48a8c27b0f954b439124e61b18e450a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:52:03.295688 9520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 60b9c6e097d0455a9c2263069fa66563. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:52:03.297338 9837 tablet_bootstrap.cc:492] T 60b9c6e097d0455a9c2263069fa66563 P b06bd78a89f042ce8df08e88f3581014: Bootstrap starting.
I20260808 16:52:03.297417 9838 tablet_bootstrap.cc:492] T 60b9c6e097d0455a9c2263069fa66563 P 4f4d7ace52364ea0915721a36afc6e37: Bootstrap starting.
I20260808 16:52:03.297950 9838 tablet_bootstrap.cc:654] T 60b9c6e097d0455a9c2263069fa66563 P 4f4d7ace52364ea0915721a36afc6e37: Neither blocks nor log segments found. Creating new log.
I20260808 16:52:03.298177 9837 tablet_bootstrap.cc:654] T 60b9c6e097d0455a9c2263069fa66563 P b06bd78a89f042ce8df08e88f3581014: Neither blocks nor log segments found. Creating new log.
I20260808 16:52:03.298259 9840 tablet_bootstrap.cc:492] T 60b9c6e097d0455a9c2263069fa66563 P fb0f7086a3144f70abc4fdc93c1be0c3: Bootstrap starting.
I20260808 16:52:03.298727 9838 log.cc:824] T 60b9c6e097d0455a9c2263069fa66563 P 4f4d7ace52364ea0915721a36afc6e37: Log is configured to *not* fsync() on all Append() calls
I20260808 16:52:03.298808 9837 log.cc:824] T 60b9c6e097d0455a9c2263069fa66563 P b06bd78a89f042ce8df08e88f3581014: Log is configured to *not* fsync() on all Append() calls
I20260808 16:52:03.298967 9840 tablet_bootstrap.cc:654] T 60b9c6e097d0455a9c2263069fa66563 P fb0f7086a3144f70abc4fdc93c1be0c3: Neither blocks nor log segments found. Creating new log.
I20260808 16:52:03.299479 9840 log.cc:824] T 60b9c6e097d0455a9c2263069fa66563 P fb0f7086a3144f70abc4fdc93c1be0c3: Log is configured to *not* fsync() on all Append() calls
I20260808 16:52:03.299508 9838 tablet_bootstrap.cc:492] T 60b9c6e097d0455a9c2263069fa66563 P 4f4d7ace52364ea0915721a36afc6e37: No bootstrap required, opened a new log
I20260808 16:52:03.299567 9838 ts_tablet_manager.cc:1397] T 60b9c6e097d0455a9c2263069fa66563 P 4f4d7ace52364ea0915721a36afc6e37: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 16:52:03.300179 9840 tablet_bootstrap.cc:492] T 60b9c6e097d0455a9c2263069fa66563 P fb0f7086a3144f70abc4fdc93c1be0c3: No bootstrap required, opened a new log
I20260808 16:52:03.300248 9840 ts_tablet_manager.cc:1397] T 60b9c6e097d0455a9c2263069fa66563 P fb0f7086a3144f70abc4fdc93c1be0c3: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 16:52:03.301409 9838 raft_consensus.cc:348] T 60b9c6e097d0455a9c2263069fa66563 P 4f4d7ace52364ea0915721a36afc6e37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb0f7086a3144f70abc4fdc93c1be0c3" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39573 } } peers { permanent_uuid: "b06bd78a89f042ce8df08e88f3581014" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 36327 } } peers { permanent_uuid: "4f4d7ace52364ea0915721a36afc6e37" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 35155 } }
I20260808 16:52:03.301595 9838 raft_consensus.cc:374] T 60b9c6e097d0455a9c2263069fa66563 P 4f4d7ace52364ea0915721a36afc6e37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:52:03.301638 9838 raft_consensus.cc:729] T 60b9c6e097d0455a9c2263069fa66563 P 4f4d7ace52364ea0915721a36afc6e37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f4d7ace52364ea0915721a36afc6e37, State: Initialized, Role: FOLLOWER
I20260808 16:52:03.301745 9838 consensus_queue.cc:260] T 60b9c6e097d0455a9c2263069fa66563 P 4f4d7ace52364ea0915721a36afc6e37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb0f7086a3144f70abc4fdc93c1be0c3" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39573 } } peers { permanent_uuid: "b06bd78a89f042ce8df08e88f3581014" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 36327 } } peers { permanent_uuid: "4f4d7ace52364ea0915721a36afc6e37" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 35155 } }
I20260808 16:52:03.302006 9838 ts_tablet_manager.cc:1428] T 60b9c6e097d0455a9c2263069fa66563 P 4f4d7ace52364ea0915721a36afc6e37: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 16:52:03.302085 9420 heartbeater.cc:503] Master 127.8.24.62:41511 was elected leader, sending a full tablet report...
I20260808 16:52:03.302121 9840 raft_consensus.cc:348] T 60b9c6e097d0455a9c2263069fa66563 P fb0f7086a3144f70abc4fdc93c1be0c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb0f7086a3144f70abc4fdc93c1be0c3" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39573 } } peers { permanent_uuid: "b06bd78a89f042ce8df08e88f3581014" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 36327 } } peers { permanent_uuid: "4f4d7ace52364ea0915721a36afc6e37" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 35155 } }
I20260808 16:52:03.302268 9840 raft_consensus.cc:374] T 60b9c6e097d0455a9c2263069fa66563 P fb0f7086a3144f70abc4fdc93c1be0c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:52:03.302289 9840 raft_consensus.cc:729] T 60b9c6e097d0455a9c2263069fa66563 P fb0f7086a3144f70abc4fdc93c1be0c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb0f7086a3144f70abc4fdc93c1be0c3, State: Initialized, Role: FOLLOWER
I20260808 16:52:03.302395 9840 consensus_queue.cc:260] T 60b9c6e097d0455a9c2263069fa66563 P fb0f7086a3144f70abc4fdc93c1be0c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb0f7086a3144f70abc4fdc93c1be0c3" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39573 } } peers { permanent_uuid: "b06bd78a89f042ce8df08e88f3581014" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 36327 } } peers { permanent_uuid: "4f4d7ace52364ea0915721a36afc6e37" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 35155 } }
I20260808 16:52:03.302528 9837 tablet_bootstrap.cc:492] T 60b9c6e097d0455a9c2263069fa66563 P b06bd78a89f042ce8df08e88f3581014: No bootstrap required, opened a new log
I20260808 16:52:03.302588 9837 ts_tablet_manager.cc:1397] T 60b9c6e097d0455a9c2263069fa66563 P b06bd78a89f042ce8df08e88f3581014: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260808 16:52:03.302753 9840 ts_tablet_manager.cc:1428] T 60b9c6e097d0455a9c2263069fa66563 P fb0f7086a3144f70abc4fdc93c1be0c3: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
I20260808 16:52:03.304100 9837 raft_consensus.cc:348] T 60b9c6e097d0455a9c2263069fa66563 P b06bd78a89f042ce8df08e88f3581014 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb0f7086a3144f70abc4fdc93c1be0c3" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39573 } } peers { permanent_uuid: "b06bd78a89f042ce8df08e88f3581014" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 36327 } } peers { permanent_uuid: "4f4d7ace52364ea0915721a36afc6e37" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 35155 } }
I20260808 16:52:03.304221 9837 raft_consensus.cc:374] T 60b9c6e097d0455a9c2263069fa66563 P b06bd78a89f042ce8df08e88f3581014 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:52:03.304251 9837 raft_consensus.cc:729] T 60b9c6e097d0455a9c2263069fa66563 P b06bd78a89f042ce8df08e88f3581014 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b06bd78a89f042ce8df08e88f3581014, State: Initialized, Role: FOLLOWER
I20260808 16:52:03.304354 9837 consensus_queue.cc:260] T 60b9c6e097d0455a9c2263069fa66563 P b06bd78a89f042ce8df08e88f3581014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb0f7086a3144f70abc4fdc93c1be0c3" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39573 } } peers { permanent_uuid: "b06bd78a89f042ce8df08e88f3581014" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 36327 } } peers { permanent_uuid: "4f4d7ace52364ea0915721a36afc6e37" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 35155 } }
I20260808 16:52:03.304584 9837 ts_tablet_manager.cc:1428] T 60b9c6e097d0455a9c2263069fa66563 P b06bd78a89f042ce8df08e88f3581014: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 16:52:03.304608 9820 heartbeater.cc:503] Master 127.8.24.62:41511 was elected leader, sending a full tablet report...
I20260808 16:52:03.304682 9622 heartbeater.cc:503] Master 127.8.24.62:41511 was elected leader, sending a full tablet report...
I20260808 16:52:03.358719 9851 raft_consensus.cc:482] T 60b9c6e097d0455a9c2263069fa66563 P b06bd78a89f042ce8df08e88f3581014 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 16:52:03.358803 9851 raft_consensus.cc:504] T 60b9c6e097d0455a9c2263069fa66563 P b06bd78a89f042ce8df08e88f3581014 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb0f7086a3144f70abc4fdc93c1be0c3" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39573 } } peers { permanent_uuid: "b06bd78a89f042ce8df08e88f3581014" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 36327 } } peers { permanent_uuid: "4f4d7ace52364ea0915721a36afc6e37" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 35155 } }
I20260808 16:52:03.359289 9851 leader_election.cc:288] T 60b9c6e097d0455a9c2263069fa66563 P b06bd78a89f042ce8df08e88f3581014 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fb0f7086a3144f70abc4fdc93c1be0c3 (127.8.24.2:39573), 4f4d7ace52364ea0915721a36afc6e37 (127.8.24.1:35155)
I20260808 16:52:03.361888 9831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:03.359589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:58397 (local address 127.8.24.2:39573)
0808 16:52:03.359637 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:03.359639 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:03.359645 (+ 6us) server_negotiation.cc:407] Connection header received
0808 16:52:03.359663 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:03.359664 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:03.359673 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:03.359718 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:52:03.359857 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:03.360301 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:03.361238 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:03.361595 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:03.361652 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:03.361662 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:03.361799 (+ 137us) server_negotiation.cc:299] Negotiation successful
0808 16:52:03.361830 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 16:52:03.361901 9822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:03.359405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.2:39573 (local address 127.8.24.3:58397)
0808 16:52:03.359457 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:03.359468 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:03.359505 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:03.359733 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:03.359734 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:03.359736 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:52:03.359799 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:03.359800 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:03.360370 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:03.360371 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:03.361155 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:03.361157 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:03.361667 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:03.361668 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:03.361737 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:03.361740 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 16:52:03.361784 (+ 44us) client_negotiation.cc:240] Negotiation successful
0808 16:52:03.361828 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29,"mutex_wait_us":46}
I20260808 16:52:03.361896 9835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:03.359496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:36147 (local address 127.8.24.1:35155)
0808 16:52:03.359548 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:03.359550 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:03.359641 (+ 91us) server_negotiation.cc:407] Connection header received
0808 16:52:03.359692 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:03.359693 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:03.359704 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:03.359747 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:52:03.359877 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:03.360325 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:03.361242 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:03.361598 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:03.361653 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:03.361662 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:03.361805 (+ 143us) server_negotiation.cc:299] Negotiation successful
0808 16:52:03.361836 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 16:52:03.361918 9853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:03.359405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:35155 (local address 127.8.24.3:36147)
0808 16:52:03.359576 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:03.359585 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:03.359637 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:03.359765 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:03.359766 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:03.359768 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:52:03.359827 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:03.359828 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:03.360393 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:03.360394 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:03.361166 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:03.361168 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:03.361669 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:03.361670 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:03.361739 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:03.361742 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 16:52:03.361792 (+ 50us) client_negotiation.cc:240] Negotiation successful
0808 16:52:03.361835 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"mutex_wait_us":15,"thread_start_us":92,"threads_started":1}
I20260808 16:52:03.362257 9551 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60b9c6e097d0455a9c2263069fa66563" candidate_uuid: "b06bd78a89f042ce8df08e88f3581014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb0f7086a3144f70abc4fdc93c1be0c3" is_pre_election: true
I20260808 16:52:03.362453 9551 raft_consensus.cc:2454] T 60b9c6e097d0455a9c2263069fa66563 P fb0f7086a3144f70abc4fdc93c1be0c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b06bd78a89f042ce8df08e88f3581014 in term 0.
I20260808 16:52:03.362712 9658 leader_election.cc:302] T 60b9c6e097d0455a9c2263069fa66563 P b06bd78a89f042ce8df08e88f3581014 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b06bd78a89f042ce8df08e88f3581014, fb0f7086a3144f70abc4fdc93c1be0c3; no voters:
I20260808 16:52:03.362828 9851 raft_consensus.cc:2781] T 60b9c6e097d0455a9c2263069fa66563 P b06bd78a89f042ce8df08e88f3581014 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 16:52:03.362879 9851 raft_consensus.cc:482] T 60b9c6e097d0455a9c2263069fa66563 P b06bd78a89f042ce8df08e88f3581014 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 16:52:03.362895 9851 raft_consensus.cc:3037] T 60b9c6e097d0455a9c2263069fa66563 P b06bd78a89f042ce8df08e88f3581014 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:52:03.363471 9851 raft_consensus.cc:504] T 60b9c6e097d0455a9c2263069fa66563 P b06bd78a89f042ce8df08e88f3581014 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb0f7086a3144f70abc4fdc93c1be0c3" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39573 } } peers { permanent_uuid: "b06bd78a89f042ce8df08e88f3581014" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 36327 } } peers { permanent_uuid: "4f4d7ace52364ea0915721a36afc6e37" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 35155 } }
I20260808 16:52:03.363572 9851 leader_election.cc:288] T 60b9c6e097d0455a9c2263069fa66563 P b06bd78a89f042ce8df08e88f3581014 [CANDIDATE]: Term 1 election: Requested vote from peers fb0f7086a3144f70abc4fdc93c1be0c3 (127.8.24.2:39573), 4f4d7ace52364ea0915721a36afc6e37 (127.8.24.1:35155)
I20260808 16:52:03.363711 9551 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60b9c6e097d0455a9c2263069fa66563" candidate_uuid: "b06bd78a89f042ce8df08e88f3581014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb0f7086a3144f70abc4fdc93c1be0c3"
I20260808 16:52:03.363763 9352 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60b9c6e097d0455a9c2263069fa66563" candidate_uuid: "b06bd78a89f042ce8df08e88f3581014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f4d7ace52364ea0915721a36afc6e37"
I20260808 16:52:03.363787 9551 raft_consensus.cc:3037] T 60b9c6e097d0455a9c2263069fa66563 P fb0f7086a3144f70abc4fdc93c1be0c3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:52:03.363847 9352 raft_consensus.cc:3037] T 60b9c6e097d0455a9c2263069fa66563 P 4f4d7ace52364ea0915721a36afc6e37 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:52:03.364405 9352 raft_consensus.cc:2454] T 60b9c6e097d0455a9c2263069fa66563 P 4f4d7ace52364ea0915721a36afc6e37 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b06bd78a89f042ce8df08e88f3581014 in term 1.
I20260808 16:52:03.364415 9551 raft_consensus.cc:2454] T 60b9c6e097d0455a9c2263069fa66563 P fb0f7086a3144f70abc4fdc93c1be0c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b06bd78a89f042ce8df08e88f3581014 in term 1.
I20260808 16:52:03.364542 9658 leader_election.cc:302] T 60b9c6e097d0455a9c2263069fa66563 P b06bd78a89f042ce8df08e88f3581014 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b06bd78a89f042ce8df08e88f3581014, fb0f7086a3144f70abc4fdc93c1be0c3; no voters:
I20260808 16:52:03.364607 9851 raft_consensus.cc:2781] T 60b9c6e097d0455a9c2263069fa66563 P b06bd78a89f042ce8df08e88f3581014 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:52:03.364703 9851 raft_consensus.cc:686] T 60b9c6e097d0455a9c2263069fa66563 P b06bd78a89f042ce8df08e88f3581014 [term 1 LEADER]: Becoming Leader. State: Replica: b06bd78a89f042ce8df08e88f3581014, State: Running, Role: LEADER
I20260808 16:52:03.364766 9851 consensus_queue.cc:237] T 60b9c6e097d0455a9c2263069fa66563 P b06bd78a89f042ce8df08e88f3581014 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb0f7086a3144f70abc4fdc93c1be0c3" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39573 } } peers { permanent_uuid: "b06bd78a89f042ce8df08e88f3581014" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 36327 } } peers { permanent_uuid: "4f4d7ace52364ea0915721a36afc6e37" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 35155 } }
I20260808 16:52:03.362257 9353 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60b9c6e097d0455a9c2263069fa66563" candidate_uuid: "b06bd78a89f042ce8df08e88f3581014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f4d7ace52364ea0915721a36afc6e37" is_pre_election: true
I20260808 16:52:03.364902 9353 raft_consensus.cc:2362] T 60b9c6e097d0455a9c2263069fa66563 P 4f4d7ace52364ea0915721a36afc6e37 [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate b06bd78a89f042ce8df08e88f3581014 in term 1. Re-sending same reply.
I20260808 16:52:03.365657 9136 catalog_manager.cc:5495] T 60b9c6e097d0455a9c2263069fa66563 P b06bd78a89f042ce8df08e88f3581014 reported cstate change: term changed from 0 to 1, leader changed from <none> to b06bd78a89f042ce8df08e88f3581014 (127.8.24.3). New cstate: current_term: 1 leader_uuid: "b06bd78a89f042ce8df08e88f3581014" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb0f7086a3144f70abc4fdc93c1be0c3" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b06bd78a89f042ce8df08e88f3581014" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 36327 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f4d7ace52364ea0915721a36afc6e37" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 35155 } health_report { overall_health: UNKNOWN } } }
W20260808 16:52:03.377101 9623 tablet.cc:2369] T 60b9c6e097d0455a9c2263069fa66563 P fb0f7086a3144f70abc4fdc93c1be0c3: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 08 16:52:03 dist-test-slave-6zbq krb5kdc[9075](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207922, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.24.1@KRBTEST.COM
I20260808 16:52:03.387809 9835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:03.381593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44894 (local address 127.8.24.1:35155)
0808 16:52:03.381650 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:03.381651 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:03.381737 (+ 86us) server_negotiation.cc:407] Connection header received
0808 16:52:03.381775 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:03.381776 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:03.381784 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:03.381827 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:03.382332 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:03.382803 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:03.383474 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:03.383647 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:03.383709 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:03.385716 (+ 2007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:03.385723 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:03.385723 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:03.385729 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:03.387021 (+ 1292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:03.387295 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:03.387297 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:03.387297 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:03.387344 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:03.387467 (+ 123us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:03.387467 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:03.387468 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:03.387566 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:03.387660 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:03.387694 (+ 34us) server_negotiation.cc:299] Negotiation successful
0808 16:52:03.387747 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
Aug 08 16:52:03 dist-test-slave-6zbq krb5kdc[9075](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207923, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 16:52:03 dist-test-slave-6zbq krb5kdc[9075](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207923, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.24.1@KRBTEST.COM
I20260808 16:52:03.409117 9835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:03.403415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44904 (local address 127.8.24.1:35155)
0808 16:52:03.403453 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:03.403455 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:03.403531 (+ 76us) server_negotiation.cc:407] Connection header received
0808 16:52:03.403569 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:03.403570 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:03.403577 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:03.403618 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:03.404111 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:03.404533 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:03.405153 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:03.405322 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:03.405360 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:03.407213 (+ 1853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:03.407214 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:03.407214 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:03.407220 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:03.408272 (+ 1052us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:03.408562 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:03.408563 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:03.408563 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:03.408611 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:03.408772 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:03.408772 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:03.408773 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:03.408849 (+ 76us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:03.408910 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:03.409012 (+ 102us) server_negotiation.cc:299] Negotiation successful
0808 16:52:03.409064 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
Aug 08 16:52:03 dist-test-slave-6zbq krb5kdc[9075](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207923, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 16:52:03 dist-test-slave-6zbq krb5kdc[9075](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207923, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.24.1@KRBTEST.COM
I20260808 16:52:03.428962 9835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:03.422455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44914 (local address 127.8.24.1:35155)
0808 16:52:03.422499 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:03.422501 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:03.422587 (+ 86us) server_negotiation.cc:407] Connection header received
0808 16:52:03.422643 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:03.422644 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:03.422652 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:03.422697 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:03.423138 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:03.423554 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:03.424212 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:03.424379 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:03.424430 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:03.426280 (+ 1850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:03.426282 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:03.426282 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:03.426289 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:03.427933 (+ 1644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:03.428278 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:03.428279 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:03.428279 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:03.428320 (+ 41us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:03.428513 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:03.428514 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:03.428515 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:03.428610 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:03.428690 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:03.428818 (+ 128us) server_negotiation.cc:299] Negotiation successful
0808 16:52:03.428883 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
W20260808 16:52:03.429589 9292 tablet_service.cc:2908] Rejecting scan request for tablet 60b9c6e097d0455a9c2263069fa66563: Uninitialized: clean time has not yet been initialized
I20260808 16:52:03.430611 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 9230
I20260808 16:52:03.441596 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 9426
I20260808 16:52:03.450739 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 9631
I20260808 16:52:03.456323 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 9095
2026-08-08T16:52:03Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2045 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786207919384338-8288-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 16:52:03 dist-test-slave-6zbq krb5kdc[9872](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 16:52:03 dist-test-slave-6zbq krb5kdc[9872](info): set up 2 sockets
Aug 08 16:52:03 dist-test-slave-6zbq krb5kdc[9872](info): commencing operation
krb5kdc: starting...
W20260808 16:52:04.578156 8288 mini_kdc.cc:121] Time spent starting KDC: real 1.101s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786207919384338-8288-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786207919384338-8288-0/krb5kdc/test-user.keytab.
Aug 08 16:52:04 dist-test-slave-6zbq krb5kdc[9872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207924, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T16:52:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T16:52:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.24.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
I20260808 16:52:04.686826 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786207919384338-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786207919384338-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786207919384338-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:38409
--webserver_interface=127.8.24.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:46549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:38409
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:52:04.764052 9894 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:52:04.764225 9894 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:52:04.764245 9894 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:52:04.765339 9894 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 16:52:04.765377 9894 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 16:52:04.765389 9894 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:52:04.765401 9894 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:52:04.765415 9894 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 16:52:04.765439 9894 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 16:52:04.766875 9894 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:46549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786207919384338-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786207919384338-8288-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:38409
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:38409
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.24.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9894
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786207919384338-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:52:04.767088 9894 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:52:04.767283 9894 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:52:04.771597 9904 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:04.771643 9894 server_base.cc:1034] running on GCE node
W20260808 16:52:04.771670 9902 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:04.771674 9900 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:04.771968 9894 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:52:04.772215 9894 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:52:04.773356 9894 hybrid_clock.cc:648] HybridClock initialized: now 1786207924773343 us; error 29 us; skew 500 ppm
Aug 08 16:52:04 dist-test-slave-6zbq krb5kdc[9872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207924, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:52:04.779070 9894 init.cc:377] Logged in from keytab as kudu/127.8.24.62@KRBTEST.COM (short username kudu)
I20260808 16:52:04.779450 9894 webserver.cc:460] Webserver started at http://127.8.24.62:35789/ using document root <none> and password file <none>
I20260808 16:52:04.779640 9894 fs_manager.cc:359] Metadata directory not provided
I20260808 16:52:04.779685 9894 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:52:04.779800 9894 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:52:04.780763 9894 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786207919384338-8288-0/minicluster-data/master-0/data/instance:
uuid: "20ca135193dc4539ac31f0fc9bb8b409"
format_stamp: "Formatted at 2026-08-08 16:52:04 on dist-test-slave-6zbq"
server_key: "73570a1f6c37efea052006bc93c57335"
server_key_iv: "59639db043ae57a07ce5d97422de0214"
server_key_version: "encryptionkey@0"
I20260808 16:52:04.781069 9894 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786207919384338-8288-0/minicluster-data/master-0/wal/instance:
uuid: "20ca135193dc4539ac31f0fc9bb8b409"
format_stamp: "Formatted at 2026-08-08 16:52:04 on dist-test-slave-6zbq"
server_key: "73570a1f6c37efea052006bc93c57335"
server_key_iv: "59639db043ae57a07ce5d97422de0214"
server_key_version: "encryptionkey@0"
I20260808 16:52:04.782243 9894 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 16:52:04.782907 9916 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:04.783057 9894 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:04.783136 9894 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786207919384338-8288-0/minicluster-data/master-0/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786207919384338-8288-0/minicluster-data/master-0/wal
uuid: "20ca135193dc4539ac31f0fc9bb8b409"
format_stamp: "Formatted at 2026-08-08 16:52:04 on dist-test-slave-6zbq"
server_key: "73570a1f6c37efea052006bc93c57335"
server_key_iv: "59639db043ae57a07ce5d97422de0214"
server_key_version: "encryptionkey@0"
I20260808 16:52:04.783202 9894 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786207919384338-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786207919384338-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786207919384338-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:52:04.801424 9894 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:52:04.802968 9894 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:52:04.803084 9894 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:52:04.807374 9894 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.62:38409
I20260808 16:52:04.807449 10001 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.62:38409 every 8 connection(s)
I20260808 16:52:04.807684 9894 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
I20260808 16:52:04.808287 10002 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:52:04.810361 10002 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 20ca135193dc4539ac31f0fc9bb8b409: Bootstrap starting.
I20260808 16:52:04.810598 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 9894
I20260808 16:52:04.810675 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786207919384338-8288-0/minicluster-data/master-0/wal/instance
I20260808 16:52:04.810806 8288 external_mini_cluster.cc:1428] Setting key 597d2035461dc5c02f0a2c96b9ef591f
I20260808 16:52:04.810964 10002 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 20ca135193dc4539ac31f0fc9bb8b409: Neither blocks nor log segments found. Creating new log.
I20260808 16:52:04.811286 10002 log.cc:824] T 00000000000000000000000000000000 P 20ca135193dc4539ac31f0fc9bb8b409: Log is configured to *not* fsync() on all Append() calls
I20260808 16:52:04.811973 10002 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 20ca135193dc4539ac31f0fc9bb8b409: No bootstrap required, opened a new log
I20260808 16:52:04.814306 10002 raft_consensus.cc:348] T 00000000000000000000000000000000 P 20ca135193dc4539ac31f0fc9bb8b409 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20ca135193dc4539ac31f0fc9bb8b409" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 38409 } }
I20260808 16:52:04.814455 10002 raft_consensus.cc:374] T 00000000000000000000000000000000 P 20ca135193dc4539ac31f0fc9bb8b409 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:52:04.814476 10002 raft_consensus.cc:729] T 00000000000000000000000000000000 P 20ca135193dc4539ac31f0fc9bb8b409 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20ca135193dc4539ac31f0fc9bb8b409, State: Initialized, Role: FOLLOWER
I20260808 16:52:04.814592 10002 consensus_queue.cc:260] T 00000000000000000000000000000000 P 20ca135193dc4539ac31f0fc9bb8b409 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20ca135193dc4539ac31f0fc9bb8b409" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 38409 } }
I20260808 16:52:04.814656 10002 raft_consensus.cc:388] T 00000000000000000000000000000000 P 20ca135193dc4539ac31f0fc9bb8b409 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 16:52:04.814678 10002 raft_consensus.cc:482] T 00000000000000000000000000000000 P 20ca135193dc4539ac31f0fc9bb8b409 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 16:52:04.814720 10002 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 20ca135193dc4539ac31f0fc9bb8b409 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:52:04.815197 10002 raft_consensus.cc:504] T 00000000000000000000000000000000 P 20ca135193dc4539ac31f0fc9bb8b409 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20ca135193dc4539ac31f0fc9bb8b409" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 38409 } }
I20260808 16:52:04.815292 10002 leader_election.cc:302] T 00000000000000000000000000000000 P 20ca135193dc4539ac31f0fc9bb8b409 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 20ca135193dc4539ac31f0fc9bb8b409; no voters:
I20260808 16:52:04.815449 10002 leader_election.cc:288] T 00000000000000000000000000000000 P 20ca135193dc4539ac31f0fc9bb8b409 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 16:52:04.815470 10010 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 20ca135193dc4539ac31f0fc9bb8b409 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:52:04.815591 10010 raft_consensus.cc:686] T 00000000000000000000000000000000 P 20ca135193dc4539ac31f0fc9bb8b409 [term 1 LEADER]: Becoming Leader. State: Replica: 20ca135193dc4539ac31f0fc9bb8b409, State: Running, Role: LEADER
I20260808 16:52:04.815668 10010 consensus_queue.cc:237] T 00000000000000000000000000000000 P 20ca135193dc4539ac31f0fc9bb8b409 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20ca135193dc4539ac31f0fc9bb8b409" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 38409 } }
I20260808 16:52:04.815953 10002 sys_catalog.cc:564] T 00000000000000000000000000000000 P 20ca135193dc4539ac31f0fc9bb8b409 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 16:52:04.815959 10011 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20ca135193dc4539ac31f0fc9bb8b409 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "20ca135193dc4539ac31f0fc9bb8b409" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20ca135193dc4539ac31f0fc9bb8b409" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 38409 } } }
I20260808 16:52:04.816067 10011 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20ca135193dc4539ac31f0fc9bb8b409 [sys.catalog]: This master's current role is: LEADER
I20260808 16:52:04.815966 10013 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20ca135193dc4539ac31f0fc9bb8b409 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 20ca135193dc4539ac31f0fc9bb8b409. Latest consensus state: current_term: 1 leader_uuid: "20ca135193dc4539ac31f0fc9bb8b409" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20ca135193dc4539ac31f0fc9bb8b409" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 38409 } } }
I20260808 16:52:04.816251 10013 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20ca135193dc4539ac31f0fc9bb8b409 [sys.catalog]: This master's current role is: LEADER
Aug 08 16:52:04 dist-test-slave-6zbq krb5kdc[9872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207924, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
W20260808 16:52:04.817623 10037 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 20ca135193dc4539ac31f0fc9bb8b409: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 16:52:04.817677 10037 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 16:52:04.817731 10019 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 16:52:04.817902 10019 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 16:52:04.819376 10019 catalog_manager.cc:1347] Generated new cluster ID: 5340558d19284ca2aa701a6eca763e3d
I20260808 16:52:04.819427 10019 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 16:52:04.821344 10007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:04.811208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53164 (local address 127.8.24.62:38409)
0808 16:52:04.811392 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:04.811401 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:04.811433 (+ 32us) server_negotiation.cc:407] Connection header received
0808 16:52:04.811891 (+ 458us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:04.811900 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:04.811957 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:04.812102 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:04.812638 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:04.813978 (+ 1340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:04.814828 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:04.815224 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:04.815321 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:04.817845 (+ 2524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:04.817849 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:04.817850 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:04.817861 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:04.819660 (+ 1799us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:04.820018 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:04.820021 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:04.820021 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:04.820106 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:04.820266 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:04.820267 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:04.820267 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:04.820510 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:04.820639 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:04.821024 (+ 385us) server_negotiation.cc:299] Negotiation successful
0808 16:52:04.821151 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":84,"threads_started":1}
I20260808 16:52:04.832621 10019 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 16:52:04.833066 10019 catalog_manager.cc:1504] Loading token signing keys...
I20260808 16:52:04.839221 10019 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 20ca135193dc4539ac31f0fc9bb8b409: Generated new TSK 0
I20260808 16:52:04.839351 10019 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.24.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.1.keytab.
Entry for principal kudu/127.8.24.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.1.keytab.
I20260808 16:52:04.864470 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786207919384338-8288-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786207919384338-8288-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786207919384338-8288-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786207919384338-8288-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.1:0
--local_ip_for_outbound_sockets=127.8.24.1
--webserver_interface=127.8.24.1
--webserver_port=0
--tserver_master_addrs=127.8.24.62:38409
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.1.keytab
--principal=kudu/127.8.24.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:46549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:52:04.941335 10047 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:52:04.941514 10047 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:52:04.941543 10047 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:52:04.942670 10047 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:52:04.942706 10047 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:52:04.942734 10047 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 16:52:04.942757 10047 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.1
I20260808 16:52:04.944141 10047 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:46549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786207919384338-8288-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786207919384338-8288-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.1.keytab
--principal=kudu/127.8.24.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786207919384338-8288-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.24.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.24.62:38409
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10047
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.1
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786207919384338-8288-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:52:04.944366 10047 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:52:04.944564 10047 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:52:04.949128 10053 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:04.949133 10058 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:04.949186 10055 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:04.949321 10047 server_base.cc:1034] running on GCE node
I20260808 16:52:04.949520 10047 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:52:04.949745 10047 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:52:04.950879 10047 hybrid_clock.cc:648] HybridClock initialized: now 1786207924950860 us; error 33 us; skew 500 ppm
Aug 08 16:52:04 dist-test-slave-6zbq krb5kdc[9872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207924, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:52:04.953066 10047 init.cc:377] Logged in from keytab as kudu/127.8.24.1@KRBTEST.COM (short username kudu)
I20260808 16:52:04.953451 10047 webserver.cc:460] Webserver started at http://127.8.24.1:37261/ using document root <none> and password file <none>
I20260808 16:52:04.953652 10047 fs_manager.cc:359] Metadata directory not provided
I20260808 16:52:04.953701 10047 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:52:04.953817 10047 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:52:04.954787 10047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786207919384338-8288-0/minicluster-data/ts-0/data/instance:
uuid: "db59e6abd91d4ebf8120160ba949b1cc"
format_stamp: "Formatted at 2026-08-08 16:52:04 on dist-test-slave-6zbq"
server_key: "26489962374c6eee3a4458d2f09769f3"
server_key_iv: "119e01d6dbdb5b90b036370fb08c62ef"
server_key_version: "encryptionkey@0"
I20260808 16:52:04.955089 10047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786207919384338-8288-0/minicluster-data/ts-0/wal/instance:
uuid: "db59e6abd91d4ebf8120160ba949b1cc"
format_stamp: "Formatted at 2026-08-08 16:52:04 on dist-test-slave-6zbq"
server_key: "26489962374c6eee3a4458d2f09769f3"
server_key_iv: "119e01d6dbdb5b90b036370fb08c62ef"
server_key_version: "encryptionkey@0"
I20260808 16:52:04.956210 10047 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 16:52:04.956836 10068 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:04.956975 10047 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 16:52:04.957053 10047 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786207919384338-8288-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786207919384338-8288-0/minicluster-data/ts-0/wal
uuid: "db59e6abd91d4ebf8120160ba949b1cc"
format_stamp: "Formatted at 2026-08-08 16:52:04 on dist-test-slave-6zbq"
server_key: "26489962374c6eee3a4458d2f09769f3"
server_key_iv: "119e01d6dbdb5b90b036370fb08c62ef"
server_key_version: "encryptionkey@0"
I20260808 16:52:04.957109 10047 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786207919384338-8288-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786207919384338-8288-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786207919384338-8288-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:52:04.970484 10047 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:52:04.971979 10047 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:52:04.972100 10047 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:52:04.972524 10047 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:52:04.972559 10047 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:04.972585 10047 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:52:04.972601 10047 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 16:52:04 dist-test-slave-6zbq krb5kdc[9872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207924, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.1@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260808 16:52:04.979247 10047 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.1:43441
I20260808 16:52:04.979305 10237 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.1:43441 every 8 connection(s)
I20260808 16:52:04.979555 10047 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786207919384338-8288-0/minicluster-data/ts-0/data/info.pb
I20260808 16:52:04.981889 10007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:04.973028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:34951 (local address 127.8.24.62:38409)
0808 16:52:04.973089 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:04.973091 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:04.973258 (+ 167us) server_negotiation.cc:407] Connection header received
0808 16:52:04.973829 (+ 571us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:04.973830 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:04.973840 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:04.973878 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:04.974644 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:04.975144 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:04.976536 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:04.976746 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:04.976803 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:04.979294 (+ 2491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:04.979295 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:04.979296 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:04.979304 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:04.980652 (+ 1348us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:04.980997 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:04.980999 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:04.980999 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:04.981057 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:04.981287 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:04.981288 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:04.981289 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:04.981399 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:04.981452 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:04.981702 (+ 250us) server_negotiation.cc:299] Negotiation successful
0808 16:52:04.981798 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 16:52:04.981997 10082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:04.972979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:38409 (local address 127.8.24.1:34951)
0808 16:52:04.973135 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:04.973177 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:04.973719 (+ 542us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:04.973945 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:04.973949 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:04.974270 (+ 321us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:04.974557 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:04.974564 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:04.975231 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:04.975232 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:04.976459 (+ 1227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:04.976462 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:04.976845 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:04.976846 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:04.976960 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:04.977376 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:04.977384 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:04.979194 (+ 1810us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:52:04.980725 (+ 1531us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:04.980732 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:04.980734 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:04.980930 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:04.981119 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:04.981119 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:04.981120 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:04.981220 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:04.981453 (+ 233us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:52:04.981460 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:52:04.981581 (+ 121us) client_negotiation.cc:769] Sending connection context
0808 16:52:04.981669 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 16:52:04.981807 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":65,"threads_started":1}
I20260808 16:52:04.982498 10238 heartbeater.cc:347] Connected to a master server at 127.8.24.62:38409
I20260808 16:52:04.982574 10238 heartbeater.cc:464] Registering TS with master...
I20260808 16:52:04.982728 10238 heartbeater.cc:511] Master 127.8.24.62:38409 requested a full tablet report, sending...
I20260808 16:52:04.983261 9934 ts_manager.cc:194] Registered new tserver with Master: db59e6abd91d4ebf8120160ba949b1cc (127.8.24.1:43441)
I20260808 16:52:04.984118 9934 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.1@KRBTEST.COM'} at 127.8.24.1:34951
I20260808 16:52:04.987974 10007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:04.983482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:50491 (local address 127.8.24.62:38409)
0808 16:52:04.983528 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:04.983530 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:04.983590 (+ 60us) server_negotiation.cc:407] Connection header received
0808 16:52:04.983654 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:04.983655 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:04.983662 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:04.983701 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:04.983992 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:04.984552 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:04.985200 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:04.985374 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:04.985435 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:04.986195 (+ 760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:04.986195 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:04.986196 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:04.986200 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:04.987239 (+ 1039us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:04.987497 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:04.987497 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:04.987497 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:04.987536 (+ 39us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:04.987648 (+ 112us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:04.987648 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:04.987649 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:04.987725 (+ 76us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:04.987773 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:04.987873 (+ 100us) server_negotiation.cc:299] Negotiation successful
0808 16:52:04.987922 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 16:52:04.987982 10248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:04.983421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:38409 (local address 127.8.24.1:50491)
0808 16:52:04.983536 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:04.983545 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:04.983603 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:04.983705 (+ 102us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:04.983706 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:04.983876 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:04.983941 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:04.983942 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:04.984644 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:04.984644 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:04.985136 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:04.985138 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:04.985476 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:04.985477 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:04.985540 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:04.985876 (+ 336us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:04.985880 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:04.986138 (+ 258us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:52:04.987300 (+ 1162us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:04.987301 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:04.987301 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:04.987444 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:04.987571 (+ 127us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:04.987571 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:04.987572 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:04.987618 (+ 46us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:04.987773 (+ 155us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:52:04.987773 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:52:04.987812 (+ 39us) client_negotiation.cc:769] Sending connection context
0808 16:52:04.987857 (+ 45us) client_negotiation.cc:240] Negotiation successful
0808 16:52:04.987920 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":66,"threads_started":1}
I20260808 16:52:04.988763 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 10047
I20260808 16:52:04.988849 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786207919384338-8288-0/minicluster-data/ts-0/wal/instance
I20260808 16:52:04.988996 8288 external_mini_cluster.cc:1428] Setting key 0c62b3481d6644c4106e72f8dabd43d9
WARNING: no policy specified for kudu/127.8.24.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.2.keytab.
Entry for principal kudu/127.8.24.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.2.keytab.
I20260808 16:52:05.010936 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786207919384338-8288-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786207919384338-8288-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786207919384338-8288-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786207919384338-8288-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.2:0
--local_ip_for_outbound_sockets=127.8.24.2
--webserver_interface=127.8.24.2
--webserver_port=0
--tserver_master_addrs=127.8.24.62:38409
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.2.keytab
--principal=kudu/127.8.24.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:46549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:52:05.087707 10251 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:52:05.087872 10251 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:52:05.087894 10251 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:52:05.088950 10251 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:52:05.088984 10251 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:52:05.089002 10251 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 16:52:05.089017 10251 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.2
I20260808 16:52:05.090376 10251 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:46549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786207919384338-8288-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786207919384338-8288-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.2.keytab
--principal=kudu/127.8.24.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786207919384338-8288-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.24.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.24.62:38409
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10251
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.2
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786207919384338-8288-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:52:05.090543 10251 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:52:05.090708 10251 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:52:05.095116 10265 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:05.095140 10258 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:05.095139 10259 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:05.095304 10251 server_base.cc:1034] running on GCE node
I20260808 16:52:05.095466 10251 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:52:05.095657 10251 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:52:05.096791 10251 hybrid_clock.cc:648] HybridClock initialized: now 1786207925096781 us; error 27 us; skew 500 ppm
Aug 08 16:52:05 dist-test-slave-6zbq krb5kdc[9872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207925, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:52:05.098946 10251 init.cc:377] Logged in from keytab as kudu/127.8.24.2@KRBTEST.COM (short username kudu)
I20260808 16:52:05.099341 10251 webserver.cc:460] Webserver started at http://127.8.24.2:36273/ using document root <none> and password file <none>
I20260808 16:52:05.099538 10251 fs_manager.cc:359] Metadata directory not provided
I20260808 16:52:05.099583 10251 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:52:05.099704 10251 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:52:05.100610 10251 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786207919384338-8288-0/minicluster-data/ts-1/data/instance:
uuid: "b84095e166154f4b93e54a628dca7381"
format_stamp: "Formatted at 2026-08-08 16:52:05 on dist-test-slave-6zbq"
server_key: "3c54c59d25667c84fcb4c28c500178cb"
server_key_iv: "4e5776332a110474e0664c2682919718"
server_key_version: "encryptionkey@0"
I20260808 16:52:05.100919 10251 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786207919384338-8288-0/minicluster-data/ts-1/wal/instance:
uuid: "b84095e166154f4b93e54a628dca7381"
format_stamp: "Formatted at 2026-08-08 16:52:05 on dist-test-slave-6zbq"
server_key: "3c54c59d25667c84fcb4c28c500178cb"
server_key_iv: "4e5776332a110474e0664c2682919718"
server_key_version: "encryptionkey@0"
I20260808 16:52:05.102084 10251 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 16:52:05.102772 10279 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:05.102914 10251 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:05.102993 10251 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786207919384338-8288-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786207919384338-8288-0/minicluster-data/ts-1/wal
uuid: "b84095e166154f4b93e54a628dca7381"
format_stamp: "Formatted at 2026-08-08 16:52:05 on dist-test-slave-6zbq"
server_key: "3c54c59d25667c84fcb4c28c500178cb"
server_key_iv: "4e5776332a110474e0664c2682919718"
server_key_version: "encryptionkey@0"
I20260808 16:52:05.103055 10251 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786207919384338-8288-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786207919384338-8288-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786207919384338-8288-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:52:05.127074 10251 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:52:05.128628 10251 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:52:05.128757 10251 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:52:05.129187 10251 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:52:05.129227 10251 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:05.129254 10251 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:52:05.129272 10251 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 16:52:05 dist-test-slave-6zbq krb5kdc[9872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207925, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.2@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260808 16:52:05.135756 10251 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.2:41153
I20260808 16:52:05.135823 10457 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.2:41153 every 8 connection(s)
I20260808 16:52:05.136117 10251 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786207919384338-8288-0/minicluster-data/ts-1/data/info.pb
I20260808 16:52:05.138203 10007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:05.129591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:58507 (local address 127.8.24.62:38409)
0808 16:52:05.129632 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:05.129635 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:05.129840 (+ 205us) server_negotiation.cc:407] Connection header received
0808 16:52:05.130285 (+ 445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:05.130287 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:05.130300 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:05.130363 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:05.130974 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:05.131557 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:05.132527 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:05.132835 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:05.132893 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:05.135900 (+ 3007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:05.135902 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:05.135903 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:05.135910 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:05.137229 (+ 1319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:05.137552 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:05.137552 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:05.137552 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:05.137597 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:05.137765 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:05.137765 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:05.137765 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:05.137855 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:05.137898 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:05.138077 (+ 179us) server_negotiation.cc:299] Negotiation successful
0808 16:52:05.138131 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 16:52:05.138389 10292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:05.129588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:38409 (local address 127.8.24.2:58507)
0808 16:52:05.129722 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:05.129762 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:05.130189 (+ 427us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:05.130427 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:05.130431 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:05.130680 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:05.130881 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:05.130886 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:05.131640 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:05.131640 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:05.132438 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:05.132440 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:05.132921 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:05.132922 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:05.133045 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:05.133521 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:05.133532 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:05.135813 (+ 2281us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:52:05.137292 (+ 1479us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:05.137297 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:05.137299 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:05.137497 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:05.137639 (+ 142us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:05.137640 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:05.137640 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:05.137723 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:05.137895 (+ 172us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:52:05.137897 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:52:05.137996 (+ 99us) client_negotiation.cc:769] Sending connection context
0808 16:52:05.138065 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 16:52:05.138171 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":81,"threads_started":1}
I20260808 16:52:05.138818 10461 heartbeater.cc:347] Connected to a master server at 127.8.24.62:38409
I20260808 16:52:05.138895 10461 heartbeater.cc:464] Registering TS with master...
I20260808 16:52:05.139063 10461 heartbeater.cc:511] Master 127.8.24.62:38409 requested a full tablet report, sending...
I20260808 16:52:05.139393 9939 ts_manager.cc:194] Registered new tserver with Master: b84095e166154f4b93e54a628dca7381 (127.8.24.2:41153)
I20260808 16:52:05.139837 9939 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.2@KRBTEST.COM'} at 127.8.24.2:58507
I20260808 16:52:05.144294 10468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:05.139746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:38409 (local address 127.8.24.2:45405)
0808 16:52:05.139880 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:05.139891 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:05.139955 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:05.140079 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:05.140079 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:05.140272 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:05.140339 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:05.140341 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:05.140920 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:05.140921 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:05.141397 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:05.141399 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:05.141661 (+ 262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:05.141662 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:05.141710 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:05.142059 (+ 349us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:05.142064 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:05.142313 (+ 249us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:52:05.143546 (+ 1233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:05.143546 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:05.143547 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:05.143706 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:05.143845 (+ 139us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:05.143846 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:05.143846 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:05.143889 (+ 43us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:05.144083 (+ 194us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:52:05.144084 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:52:05.144113 (+ 29us) client_negotiation.cc:769] Sending connection context
0808 16:52:05.144162 (+ 49us) client_negotiation.cc:240] Negotiation successful
0808 16:52:05.144230 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":70,"threads_started":1}
I20260808 16:52:05.144299 10007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:05.139848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:45405 (local address 127.8.24.62:38409)
0808 16:52:05.139889 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:05.139891 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:05.139944 (+ 53us) server_negotiation.cc:407] Connection header received
0808 16:52:05.140014 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:05.140014 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:05.140024 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:05.140070 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:05.140414 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:05.140848 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:05.141452 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:05.141614 (+ 162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:05.141653 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:05.142491 (+ 838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:05.142493 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:05.142494 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:05.142503 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:05.143481 (+ 978us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:05.143762 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:05.143763 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:05.143763 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:05.143804 (+ 41us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:05.143928 (+ 124us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:05.143928 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:05.143928 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:05.144043 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:05.144087 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:05.144179 (+ 92us) server_negotiation.cc:299] Negotiation successful
0808 16:52:05.144245 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 16:52:05.145081 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 10251
I20260808 16:52:05.145155 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786207919384338-8288-0/minicluster-data/ts-1/wal/instance
I20260808 16:52:05.145328 8288 external_mini_cluster.cc:1428] Setting key 167eefb70f4c56aed69ee8a67a2b52e1
WARNING: no policy specified for kudu/127.8.24.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.3.keytab.
Entry for principal kudu/127.8.24.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.3.keytab.
I20260808 16:52:05.166900 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786207919384338-8288-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786207919384338-8288-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786207919384338-8288-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786207919384338-8288-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.3:0
--local_ip_for_outbound_sockets=127.8.24.3
--webserver_interface=127.8.24.3
--webserver_port=0
--tserver_master_addrs=127.8.24.62:38409
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.3.keytab
--principal=kudu/127.8.24.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:46549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:52:05.246745 10473 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:52:05.246932 10473 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:52:05.246954 10473 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:52:05.248086 10473 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:52:05.248124 10473 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:52:05.248143 10473 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 16:52:05.248169 10473 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.3
I20260808 16:52:05.249548 10473 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:46549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786207919384338-8288-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786207919384338-8288-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.3.keytab
--principal=kudu/127.8.24.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786207919384338-8288-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.24.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.24.62:38409
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10473
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.3
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786207919384338-8288-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:52:05.249758 10473 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:52:05.249956 10473 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:52:05.254520 10483 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:05.254578 10481 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:05.254508 10485 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:05.254735 10473 server_base.cc:1034] running on GCE node
I20260808 16:52:05.254968 10473 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:52:05.255213 10473 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:52:05.256353 10473 hybrid_clock.cc:648] HybridClock initialized: now 1786207925256333 us; error 33 us; skew 500 ppm
Aug 08 16:52:05 dist-test-slave-6zbq krb5kdc[9872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207925, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:52:05.258745 10473 init.cc:377] Logged in from keytab as kudu/127.8.24.3@KRBTEST.COM (short username kudu)
I20260808 16:52:05.260547 10473 webserver.cc:460] Webserver started at http://127.8.24.3:45923/ using document root <none> and password file <none>
I20260808 16:52:05.260767 10473 fs_manager.cc:359] Metadata directory not provided
I20260808 16:52:05.260825 10473 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:52:05.260949 10473 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:52:05.261924 10473 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786207919384338-8288-0/minicluster-data/ts-2/data/instance:
uuid: "27474c2149a74ec3a092d6f0abced2d2"
format_stamp: "Formatted at 2026-08-08 16:52:05 on dist-test-slave-6zbq"
server_key: "f0b5588698caf35f656096cccc255bff"
server_key_iv: "77175e548dadc8d022d92cfd1858e36a"
server_key_version: "encryptionkey@0"
I20260808 16:52:05.262240 10473 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786207919384338-8288-0/minicluster-data/ts-2/wal/instance:
uuid: "27474c2149a74ec3a092d6f0abced2d2"
format_stamp: "Formatted at 2026-08-08 16:52:05 on dist-test-slave-6zbq"
server_key: "f0b5588698caf35f656096cccc255bff"
server_key_iv: "77175e548dadc8d022d92cfd1858e36a"
server_key_version: "encryptionkey@0"
I20260808 16:52:05.263417 10473 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 16:52:05.264029 10497 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:05.264185 10473 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 16:52:05.264266 10473 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786207919384338-8288-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786207919384338-8288-0/minicluster-data/ts-2/wal
uuid: "27474c2149a74ec3a092d6f0abced2d2"
format_stamp: "Formatted at 2026-08-08 16:52:05 on dist-test-slave-6zbq"
server_key: "f0b5588698caf35f656096cccc255bff"
server_key_iv: "77175e548dadc8d022d92cfd1858e36a"
server_key_version: "encryptionkey@0"
I20260808 16:52:05.264318 10473 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786207919384338-8288-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786207919384338-8288-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786207919384338-8288-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:52:05.276861 10473 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:52:05.278505 10473 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:52:05.278605 10473 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:52:05.279017 10473 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:52:05.279055 10473 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:05.279075 10473 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:52:05.279088 10473 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 16:52:05 dist-test-slave-6zbq krb5kdc[9872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207925, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.3@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260808 16:52:05.285713 10473 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.3:36397
I20260808 16:52:05.286037 10667 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.3:36397 every 8 connection(s)
I20260808 16:52:05.286103 10473 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786207919384338-8288-0/minicluster-data/ts-2/data/info.pb
I20260808 16:52:05.287978 10007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:05.279434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:40505 (local address 127.8.24.62:38409)
0808 16:52:05.279496 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:05.279498 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:05.279678 (+ 180us) server_negotiation.cc:407] Connection header received
0808 16:52:05.280097 (+ 419us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:05.280098 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:05.280111 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:05.280188 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:05.280932 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:05.281672 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:05.283026 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:05.283313 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:05.283398 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:05.285459 (+ 2061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:05.285460 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:05.285461 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:05.285468 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:05.286905 (+ 1437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:05.287313 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:05.287314 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:05.287315 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:05.287366 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:05.287543 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:05.287543 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:05.287544 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:05.287637 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:05.287678 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:05.287863 (+ 185us) server_negotiation.cc:299] Negotiation successful
0808 16:52:05.287919 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 16:52:05.288137 10510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:05.279397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:38409 (local address 127.8.24.3:40505)
0808 16:52:05.279559 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:05.279596 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:05.280014 (+ 418us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:05.280266 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:05.280271 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:05.280555 (+ 284us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:05.280838 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:05.280844 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:05.281795 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:05.281796 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:05.282951 (+ 1155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:05.282954 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:05.283419 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:05.283420 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:05.283533 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:05.284013 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:05.284021 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:05.285370 (+ 1349us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:52:05.286971 (+ 1601us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:05.286976 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:05.286978 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:05.287251 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:05.287420 (+ 169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:05.287421 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:05.287421 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:05.287508 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:05.287671 (+ 163us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:52:05.287673 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:52:05.287780 (+ 107us) client_negotiation.cc:769] Sending connection context
0808 16:52:05.287851 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 16:52:05.287956 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":93,"threads_started":1}
I20260808 16:52:05.288587 10668 heartbeater.cc:347] Connected to a master server at 127.8.24.62:38409
I20260808 16:52:05.288664 10668 heartbeater.cc:464] Registering TS with master...
I20260808 16:52:05.288825 10668 heartbeater.cc:511] Master 127.8.24.62:38409 requested a full tablet report, sending...
I20260808 16:52:05.289194 9939 ts_manager.cc:194] Registered new tserver with Master: 27474c2149a74ec3a092d6f0abced2d2 (127.8.24.3:36397)
I20260808 16:52:05.289590 9939 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.3@KRBTEST.COM'} at 127.8.24.3:40505
I20260808 16:52:05.291010 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 10473
I20260808 16:52:05.291086 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786207919384338-8288-0/minicluster-data/ts-2/wal/instance
I20260808 16:52:05.291244 8288 external_mini_cluster.cc:1428] Setting key da9f72acb2e0d9754f4abce6e60f71d5
I20260808 16:52:05.292049 8288 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 16:52:05.296455 10007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:05.289525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:58551 (local address 127.8.24.62:38409)
0808 16:52:05.289562 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:05.289563 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:05.289627 (+ 64us) server_negotiation.cc:407] Connection header received
0808 16:52:05.289679 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:05.289680 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:05.289689 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:05.289740 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:05.290070 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:05.290629 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:05.291337 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:05.291569 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:05.291627 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:05.294273 (+ 2646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:05.294274 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:05.294275 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:05.294281 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:05.295474 (+ 1193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:05.295760 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:05.295761 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:05.295761 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:05.295812 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:05.295998 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:05.295999 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:05.296000 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:05.296105 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:05.296136 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:05.296308 (+ 172us) server_negotiation.cc:299] Negotiation successful
0808 16:52:05.296370 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 16:52:05.296464 10677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:05.289458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:38409 (local address 127.8.24.3:58551)
0808 16:52:05.289576 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:05.289586 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:05.289636 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:05.289744 (+ 108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:05.289745 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:05.289939 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:05.290000 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:05.290001 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:05.290705 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:05.290706 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:05.291248 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:05.291250 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:05.291628 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:05.291629 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:05.291702 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:05.293912 (+ 2210us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:05.293917 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:05.294185 (+ 268us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:52:05.295546 (+ 1361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:05.295547 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:05.295548 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:05.295700 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:05.295864 (+ 164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:05.295865 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:05.295866 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:05.295922 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:05.296172 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:52:05.296172 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:52:05.296203 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 16:52:05.296265 (+ 62us) client_negotiation.cc:240] Negotiation successful
0808 16:52:05.296363 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":72,"threads_started":1}
Aug 08 16:52:05 dist-test-slave-6zbq krb5kdc[9872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207925, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 16:52:05 dist-test-slave-6zbq krb5kdc[9872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207925, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260808 16:52:05.313575 10007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:05.307622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53174 (local address 127.8.24.62:38409)
0808 16:52:05.307673 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:05.307675 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:05.307684 (+ 9us) server_negotiation.cc:407] Connection header received
0808 16:52:05.307739 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:05.307740 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:05.307748 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:05.307794 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:05.308245 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:05.308697 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:05.309335 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:05.309506 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:05.309548 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:05.311515 (+ 1967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:05.311516 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:05.311517 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:05.311523 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:05.312586 (+ 1063us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:05.312916 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:05.312917 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:05.312918 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:05.312967 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:05.313124 (+ 157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:05.313125 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:05.313125 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:05.313225 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:05.313317 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:05.313426 (+ 109us) server_negotiation.cc:299] Negotiation successful
0808 16:52:05.313492 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 16:52:05.314968 9939 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53174:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 16:52:05.316547 9939 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 16:52:05.321882 10694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:05.318450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.2:41153 (local address 127.0.0.1:35388)
0808 16:52:05.318571 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:05.318606 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:05.318663 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:05.318944 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:05.318947 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:05.318949 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:52:05.319005 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:05.319006 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:05.319964 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:05.319965 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:05.320886 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:05.320889 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:05.321590 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:05.321591 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:05.321677 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:05.321691 (+ 14us) client_negotiation.cc:769] Sending connection context
0808 16:52:05.321758 (+ 67us) client_negotiation.cc:240] Negotiation successful
0808 16:52:05.321804 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":78,"threads_started":1}
I20260808 16:52:05.321913 10695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:05.318590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35388 (local address 127.8.24.2:41153)
0808 16:52:05.318783 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:05.318787 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:05.318797 (+ 10us) server_negotiation.cc:407] Connection header received
0808 16:52:05.318837 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:05.318840 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:05.318857 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:05.318938 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:52:05.319077 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:05.319890 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:05.320969 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:05.321519 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:05.321586 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:05.321642 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:05.321779 (+ 137us) server_negotiation.cc:299] Negotiation successful
0808 16:52:05.321820 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":72,"threads_started":1}
I20260808 16:52:05.322685 10696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:05.318667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.3:36397 (local address 127.0.0.1:48676)
0808 16:52:05.319020 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:05.319028 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:05.319075 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:05.319339 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:05.319339 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:05.319341 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:52:05.319371 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:05.319373 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:05.320575 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:05.320576 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:05.321506 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:05.321509 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:05.322473 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:05.322474 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:05.322527 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:05.322531 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 16:52:05.322587 (+ 56us) client_negotiation.cc:240] Negotiation successful
0808 16:52:05.322622 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":38,"threads_started":1}
I20260808 16:52:05.322716 10697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:05.318762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48676 (local address 127.8.24.3:36397)
0808 16:52:05.318906 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:05.318910 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:05.319151 (+ 241us) server_negotiation.cc:407] Connection header received
0808 16:52:05.319195 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:05.319197 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:05.319210 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:05.319283 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:52:05.319440 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:05.320489 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:05.321582 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:05.322401 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:05.322471 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:05.322525 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:05.322599 (+ 74us) server_negotiation.cc:299] Negotiation successful
0808 16:52:05.322631 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":78,"threads_started":1}
I20260808 16:52:05.322858 10357 tablet_service.cc:1467] Processing CreateTablet for tablet fdeaf89526c74df58e8e61fc0f0fdea3 (DEFAULT_TABLE table=test-table [id=be152bf962c74ab1a03ae0bc0d20a980]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:52:05.323186 10357 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fdeaf89526c74df58e8e61fc0f0fdea3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:52:05.323500 10692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:05.318453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:43441 (local address 127.0.0.1:35332)
0808 16:52:05.319012 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:05.319022 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:05.319097 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:05.319327 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:05.319328 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:05.319330 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:52:05.319374 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:05.319376 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:05.321580 (+ 2204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:05.321581 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:05.322421 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:05.322424 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:05.323221 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:05.323222 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:05.323290 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:05.323293 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 16:52:05.323401 (+ 108us) client_negotiation.cc:240] Negotiation successful
0808 16:52:05.323439 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":545,"thread_start_us":68,"threads_started":1}
I20260808 16:52:05.323398 10568 tablet_service.cc:1467] Processing CreateTablet for tablet fdeaf89526c74df58e8e61fc0f0fdea3 (DEFAULT_TABLE table=test-table [id=be152bf962c74ab1a03ae0bc0d20a980]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:52:05.323698 10568 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fdeaf89526c74df58e8e61fc0f0fdea3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:52:05.323745 10698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:05.318789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35332 (local address 127.8.24.1:43441)
0808 16:52:05.318967 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:05.318971 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:05.319075 (+ 104us) server_negotiation.cc:407] Connection header received
0808 16:52:05.319182 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:05.319184 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:05.319198 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:05.319250 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:52:05.319440 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:05.320492 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:05.322509 (+ 2017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:05.323148 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:05.323211 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:05.323263 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:05.323626 (+ 363us) server_negotiation.cc:299] Negotiation successful
0808 16:52:05.323668 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":78,"threads_started":1}
I20260808 16:52:05.324235 10143 tablet_service.cc:1467] Processing CreateTablet for tablet fdeaf89526c74df58e8e61fc0f0fdea3 (DEFAULT_TABLE table=test-table [id=be152bf962c74ab1a03ae0bc0d20a980]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:52:05.324468 10143 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fdeaf89526c74df58e8e61fc0f0fdea3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:52:05.327059 10700 tablet_bootstrap.cc:492] T fdeaf89526c74df58e8e61fc0f0fdea3 P db59e6abd91d4ebf8120160ba949b1cc: Bootstrap starting.
I20260808 16:52:05.327301 10701 tablet_bootstrap.cc:492] T fdeaf89526c74df58e8e61fc0f0fdea3 P b84095e166154f4b93e54a628dca7381: Bootstrap starting.
I20260808 16:52:05.327708 10700 tablet_bootstrap.cc:654] T fdeaf89526c74df58e8e61fc0f0fdea3 P db59e6abd91d4ebf8120160ba949b1cc: Neither blocks nor log segments found. Creating new log.
I20260808 16:52:05.327880 10701 tablet_bootstrap.cc:654] T fdeaf89526c74df58e8e61fc0f0fdea3 P b84095e166154f4b93e54a628dca7381: Neither blocks nor log segments found. Creating new log.
I20260808 16:52:05.328045 10700 log.cc:824] T fdeaf89526c74df58e8e61fc0f0fdea3 P db59e6abd91d4ebf8120160ba949b1cc: Log is configured to *not* fsync() on all Append() calls
I20260808 16:52:05.328179 10701 log.cc:824] T fdeaf89526c74df58e8e61fc0f0fdea3 P b84095e166154f4b93e54a628dca7381: Log is configured to *not* fsync() on all Append() calls
I20260808 16:52:05.328511 10699 tablet_bootstrap.cc:492] T fdeaf89526c74df58e8e61fc0f0fdea3 P 27474c2149a74ec3a092d6f0abced2d2: Bootstrap starting.
I20260808 16:52:05.328773 10700 tablet_bootstrap.cc:492] T fdeaf89526c74df58e8e61fc0f0fdea3 P db59e6abd91d4ebf8120160ba949b1cc: No bootstrap required, opened a new log
I20260808 16:52:05.328833 10700 ts_tablet_manager.cc:1397] T fdeaf89526c74df58e8e61fc0f0fdea3 P db59e6abd91d4ebf8120160ba949b1cc: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 16:52:05.328907 10701 tablet_bootstrap.cc:492] T fdeaf89526c74df58e8e61fc0f0fdea3 P b84095e166154f4b93e54a628dca7381: No bootstrap required, opened a new log
I20260808 16:52:05.328974 10701 ts_tablet_manager.cc:1397] T fdeaf89526c74df58e8e61fc0f0fdea3 P b84095e166154f4b93e54a628dca7381: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 16:52:05.329396 10699 tablet_bootstrap.cc:654] T fdeaf89526c74df58e8e61fc0f0fdea3 P 27474c2149a74ec3a092d6f0abced2d2: Neither blocks nor log segments found. Creating new log.
I20260808 16:52:05.329694 10699 log.cc:824] T fdeaf89526c74df58e8e61fc0f0fdea3 P 27474c2149a74ec3a092d6f0abced2d2: Log is configured to *not* fsync() on all Append() calls
I20260808 16:52:05.330457 10699 tablet_bootstrap.cc:492] T fdeaf89526c74df58e8e61fc0f0fdea3 P 27474c2149a74ec3a092d6f0abced2d2: No bootstrap required, opened a new log
I20260808 16:52:05.330523 10699 ts_tablet_manager.cc:1397] T fdeaf89526c74df58e8e61fc0f0fdea3 P 27474c2149a74ec3a092d6f0abced2d2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 16:52:05.330531 10700 raft_consensus.cc:348] T fdeaf89526c74df58e8e61fc0f0fdea3 P db59e6abd91d4ebf8120160ba949b1cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b84095e166154f4b93e54a628dca7381" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 41153 } } peers { permanent_uuid: "db59e6abd91d4ebf8120160ba949b1cc" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 43441 } } peers { permanent_uuid: "27474c2149a74ec3a092d6f0abced2d2" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 36397 } }
I20260808 16:52:05.330686 10700 raft_consensus.cc:374] T fdeaf89526c74df58e8e61fc0f0fdea3 P db59e6abd91d4ebf8120160ba949b1cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:52:05.330725 10700 raft_consensus.cc:729] T fdeaf89526c74df58e8e61fc0f0fdea3 P db59e6abd91d4ebf8120160ba949b1cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db59e6abd91d4ebf8120160ba949b1cc, State: Initialized, Role: FOLLOWER
I20260808 16:52:05.330834 10700 consensus_queue.cc:260] T fdeaf89526c74df58e8e61fc0f0fdea3 P db59e6abd91d4ebf8120160ba949b1cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b84095e166154f4b93e54a628dca7381" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 41153 } } peers { permanent_uuid: "db59e6abd91d4ebf8120160ba949b1cc" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 43441 } } peers { permanent_uuid: "27474c2149a74ec3a092d6f0abced2d2" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 36397 } }
I20260808 16:52:05.330858 10701 raft_consensus.cc:348] T fdeaf89526c74df58e8e61fc0f0fdea3 P b84095e166154f4b93e54a628dca7381 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b84095e166154f4b93e54a628dca7381" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 41153 } } peers { permanent_uuid: "db59e6abd91d4ebf8120160ba949b1cc" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 43441 } } peers { permanent_uuid: "27474c2149a74ec3a092d6f0abced2d2" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 36397 } }
I20260808 16:52:05.331012 10701 raft_consensus.cc:374] T fdeaf89526c74df58e8e61fc0f0fdea3 P b84095e166154f4b93e54a628dca7381 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:52:05.331038 10701 raft_consensus.cc:729] T fdeaf89526c74df58e8e61fc0f0fdea3 P b84095e166154f4b93e54a628dca7381 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b84095e166154f4b93e54a628dca7381, State: Initialized, Role: FOLLOWER
I20260808 16:52:05.331077 10700 ts_tablet_manager.cc:1428] T fdeaf89526c74df58e8e61fc0f0fdea3 P db59e6abd91d4ebf8120160ba949b1cc: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 16:52:05.331149 10701 consensus_queue.cc:260] T fdeaf89526c74df58e8e61fc0f0fdea3 P b84095e166154f4b93e54a628dca7381 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b84095e166154f4b93e54a628dca7381" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 41153 } } peers { permanent_uuid: "db59e6abd91d4ebf8120160ba949b1cc" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 43441 } } peers { permanent_uuid: "27474c2149a74ec3a092d6f0abced2d2" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 36397 } }
I20260808 16:52:05.331193 10238 heartbeater.cc:503] Master 127.8.24.62:38409 was elected leader, sending a full tablet report...
I20260808 16:52:05.331413 10701 ts_tablet_manager.cc:1428] T fdeaf89526c74df58e8e61fc0f0fdea3 P b84095e166154f4b93e54a628dca7381: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 16:52:05.331444 10461 heartbeater.cc:503] Master 127.8.24.62:38409 was elected leader, sending a full tablet report...
I20260808 16:52:05.332312 10699 raft_consensus.cc:348] T fdeaf89526c74df58e8e61fc0f0fdea3 P 27474c2149a74ec3a092d6f0abced2d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b84095e166154f4b93e54a628dca7381" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 41153 } } peers { permanent_uuid: "db59e6abd91d4ebf8120160ba949b1cc" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 43441 } } peers { permanent_uuid: "27474c2149a74ec3a092d6f0abced2d2" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 36397 } }
I20260808 16:52:05.332463 10699 raft_consensus.cc:374] T fdeaf89526c74df58e8e61fc0f0fdea3 P 27474c2149a74ec3a092d6f0abced2d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:52:05.332499 10699 raft_consensus.cc:729] T fdeaf89526c74df58e8e61fc0f0fdea3 P 27474c2149a74ec3a092d6f0abced2d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27474c2149a74ec3a092d6f0abced2d2, State: Initialized, Role: FOLLOWER
I20260808 16:52:05.332592 10699 consensus_queue.cc:260] T fdeaf89526c74df58e8e61fc0f0fdea3 P 27474c2149a74ec3a092d6f0abced2d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b84095e166154f4b93e54a628dca7381" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 41153 } } peers { permanent_uuid: "db59e6abd91d4ebf8120160ba949b1cc" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 43441 } } peers { permanent_uuid: "27474c2149a74ec3a092d6f0abced2d2" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 36397 } }
I20260808 16:52:05.332778 10699 ts_tablet_manager.cc:1428] T fdeaf89526c74df58e8e61fc0f0fdea3 P 27474c2149a74ec3a092d6f0abced2d2: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260808 16:52:05.332834 10668 heartbeater.cc:503] Master 127.8.24.62:38409 was elected leader, sending a full tablet report...
W20260808 16:52:05.386682 10462 tablet.cc:2369] T fdeaf89526c74df58e8e61fc0f0fdea3 P b84095e166154f4b93e54a628dca7381: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 16:52:05.427000 10707 raft_consensus.cc:482] T fdeaf89526c74df58e8e61fc0f0fdea3 P 27474c2149a74ec3a092d6f0abced2d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 16:52:05.427080 10707 raft_consensus.cc:504] T fdeaf89526c74df58e8e61fc0f0fdea3 P 27474c2149a74ec3a092d6f0abced2d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b84095e166154f4b93e54a628dca7381" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 41153 } } peers { permanent_uuid: "db59e6abd91d4ebf8120160ba949b1cc" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 43441 } } peers { permanent_uuid: "27474c2149a74ec3a092d6f0abced2d2" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 36397 } }
I20260808 16:52:05.427345 10707 leader_election.cc:288] T fdeaf89526c74df58e8e61fc0f0fdea3 P 27474c2149a74ec3a092d6f0abced2d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b84095e166154f4b93e54a628dca7381 (127.8.24.2:41153), db59e6abd91d4ebf8120160ba949b1cc (127.8.24.1:43441)
I20260808 16:52:05.429926 10698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:05.427568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:40207 (local address 127.8.24.1:43441)
0808 16:52:05.427607 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:05.427609 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:05.427619 (+ 10us) server_negotiation.cc:407] Connection header received
0808 16:52:05.427665 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:05.427666 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:05.427677 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:05.427728 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:52:05.427873 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:05.428362 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:05.429293 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:05.429622 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:05.429678 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:05.429688 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:05.429829 (+ 141us) server_negotiation.cc:299] Negotiation successful
0808 16:52:05.429863 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 16:52:05.429927 10510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:05.427492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:43441 (local address 127.8.24.3:40207)
0808 16:52:05.427549 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:05.427561 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:05.427609 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:05.427749 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:05.427749 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:05.427751 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:52:05.427815 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:05.427817 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:05.428430 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:05.428431 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:05.429212 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:05.429214 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:05.429694 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:05.429695 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:05.429769 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:05.429772 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 16:52:05.429812 (+ 40us) client_negotiation.cc:240] Negotiation successful
0808 16:52:05.429848 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"mutex_wait_us":28}
I20260808 16:52:05.429935 10695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:05.427549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:47769 (local address 127.8.24.2:41153)
0808 16:52:05.427582 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:05.427585 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:05.427706 (+ 121us) server_negotiation.cc:407] Connection header received
0808 16:52:05.427774 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:05.427775 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:05.427787 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:05.427839 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:52:05.427966 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:05.428420 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:05.429292 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:05.429625 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:05.429682 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:05.429692 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:05.429849 (+ 157us) server_negotiation.cc:299] Negotiation successful
0808 16:52:05.429882 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 16:52:05.429948 10708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:05.427492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.2:41153 (local address 127.8.24.3:47769)
0808 16:52:05.427637 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:05.427646 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:05.427704 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:05.427840 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:05.427841 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:05.427842 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:52:05.427911 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:05.427913 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:05.428498 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:05.428498 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:05.429206 (+ 708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:05.429208 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:05.429700 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:05.429701 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:05.429782 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:05.429785 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 16:52:05.429835 (+ 50us) client_negotiation.cc:240] Negotiation successful
0808 16:52:05.429868 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"mutex_wait_us":11,"thread_start_us":74,"threads_started":1}
I20260808 16:52:05.430244 10172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdeaf89526c74df58e8e61fc0f0fdea3" candidate_uuid: "27474c2149a74ec3a092d6f0abced2d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db59e6abd91d4ebf8120160ba949b1cc" is_pre_election: true
I20260808 16:52:05.430253 10388 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdeaf89526c74df58e8e61fc0f0fdea3" candidate_uuid: "27474c2149a74ec3a092d6f0abced2d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b84095e166154f4b93e54a628dca7381" is_pre_election: true
I20260808 16:52:05.430394 10388 raft_consensus.cc:2454] T fdeaf89526c74df58e8e61fc0f0fdea3 P b84095e166154f4b93e54a628dca7381 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 27474c2149a74ec3a092d6f0abced2d2 in term 0.
I20260808 16:52:05.430397 10172 raft_consensus.cc:2454] T fdeaf89526c74df58e8e61fc0f0fdea3 P db59e6abd91d4ebf8120160ba949b1cc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 27474c2149a74ec3a092d6f0abced2d2 in term 0.
I20260808 16:52:05.430590 10503 leader_election.cc:302] T fdeaf89526c74df58e8e61fc0f0fdea3 P 27474c2149a74ec3a092d6f0abced2d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27474c2149a74ec3a092d6f0abced2d2, b84095e166154f4b93e54a628dca7381; no voters:
I20260808 16:52:05.430709 10707 raft_consensus.cc:2781] T fdeaf89526c74df58e8e61fc0f0fdea3 P 27474c2149a74ec3a092d6f0abced2d2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 16:52:05.430769 10707 raft_consensus.cc:482] T fdeaf89526c74df58e8e61fc0f0fdea3 P 27474c2149a74ec3a092d6f0abced2d2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 16:52:05.430787 10707 raft_consensus.cc:3037] T fdeaf89526c74df58e8e61fc0f0fdea3 P 27474c2149a74ec3a092d6f0abced2d2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:52:05.431396 10707 raft_consensus.cc:504] T fdeaf89526c74df58e8e61fc0f0fdea3 P 27474c2149a74ec3a092d6f0abced2d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b84095e166154f4b93e54a628dca7381" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 41153 } } peers { permanent_uuid: "db59e6abd91d4ebf8120160ba949b1cc" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 43441 } } peers { permanent_uuid: "27474c2149a74ec3a092d6f0abced2d2" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 36397 } }
I20260808 16:52:05.431572 10707 leader_election.cc:288] T fdeaf89526c74df58e8e61fc0f0fdea3 P 27474c2149a74ec3a092d6f0abced2d2 [CANDIDATE]: Term 1 election: Requested vote from peers b84095e166154f4b93e54a628dca7381 (127.8.24.2:41153), db59e6abd91d4ebf8120160ba949b1cc (127.8.24.1:43441)
I20260808 16:52:05.431607 10388 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdeaf89526c74df58e8e61fc0f0fdea3" candidate_uuid: "27474c2149a74ec3a092d6f0abced2d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b84095e166154f4b93e54a628dca7381"
I20260808 16:52:05.431679 10388 raft_consensus.cc:3037] T fdeaf89526c74df58e8e61fc0f0fdea3 P b84095e166154f4b93e54a628dca7381 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:52:05.432476 10388 raft_consensus.cc:2454] T fdeaf89526c74df58e8e61fc0f0fdea3 P b84095e166154f4b93e54a628dca7381 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 27474c2149a74ec3a092d6f0abced2d2 in term 1.
I20260808 16:52:05.432580 10172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdeaf89526c74df58e8e61fc0f0fdea3" candidate_uuid: "27474c2149a74ec3a092d6f0abced2d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db59e6abd91d4ebf8120160ba949b1cc"
I20260808 16:52:05.432644 10172 raft_consensus.cc:3037] T fdeaf89526c74df58e8e61fc0f0fdea3 P db59e6abd91d4ebf8120160ba949b1cc [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:52:05.432663 10503 leader_election.cc:302] T fdeaf89526c74df58e8e61fc0f0fdea3 P 27474c2149a74ec3a092d6f0abced2d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27474c2149a74ec3a092d6f0abced2d2, b84095e166154f4b93e54a628dca7381; no voters:
I20260808 16:52:05.432736 10707 raft_consensus.cc:2781] T fdeaf89526c74df58e8e61fc0f0fdea3 P 27474c2149a74ec3a092d6f0abced2d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:52:05.432901 10707 raft_consensus.cc:686] T fdeaf89526c74df58e8e61fc0f0fdea3 P 27474c2149a74ec3a092d6f0abced2d2 [term 1 LEADER]: Becoming Leader. State: Replica: 27474c2149a74ec3a092d6f0abced2d2, State: Running, Role: LEADER
I20260808 16:52:05.432989 10707 consensus_queue.cc:237] T fdeaf89526c74df58e8e61fc0f0fdea3 P 27474c2149a74ec3a092d6f0abced2d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b84095e166154f4b93e54a628dca7381" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 41153 } } peers { permanent_uuid: "db59e6abd91d4ebf8120160ba949b1cc" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 43441 } } peers { permanent_uuid: "27474c2149a74ec3a092d6f0abced2d2" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 36397 } }
I20260808 16:52:05.433391 10172 raft_consensus.cc:2454] T fdeaf89526c74df58e8e61fc0f0fdea3 P db59e6abd91d4ebf8120160ba949b1cc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 27474c2149a74ec3a092d6f0abced2d2 in term 1.
I20260808 16:52:05.433996 9934 catalog_manager.cc:5495] T fdeaf89526c74df58e8e61fc0f0fdea3 P 27474c2149a74ec3a092d6f0abced2d2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 27474c2149a74ec3a092d6f0abced2d2 (127.8.24.3). New cstate: current_term: 1 leader_uuid: "27474c2149a74ec3a092d6f0abced2d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b84095e166154f4b93e54a628dca7381" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 41153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db59e6abd91d4ebf8120160ba949b1cc" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 43441 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "27474c2149a74ec3a092d6f0abced2d2" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 36397 } health_report { overall_health: HEALTHY } } }
I20260808 16:52:05.465549 10697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:05.463554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48682 (local address 127.8.24.3:36397)
0808 16:52:05.463604 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:05.463605 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:05.463613 (+ 8us) server_negotiation.cc:407] Connection header received
0808 16:52:05.463653 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:05.463654 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:05.463667 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:05.463712 (+ 45us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 16:52:05.463819 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:05.464236 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:05.464904 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:05.465068 (+ 164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:05.465105 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:05.465274 (+ 169us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 16:52:05.465356 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 16:52:05.465423 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:05.465454 (+ 31us) server_negotiation.cc:299] Negotiation successful
0808 16:52:05.465471 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 16:52:05.467090 10172 raft_consensus.cc:1261] T fdeaf89526c74df58e8e61fc0f0fdea3 P db59e6abd91d4ebf8120160ba949b1cc [term 1 FOLLOWER]: Refusing update from remote peer 27474c2149a74ec3a092d6f0abced2d2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:52:05.467413 10709 consensus_queue.cc:1035] T fdeaf89526c74df58e8e61fc0f0fdea3 P 27474c2149a74ec3a092d6f0abced2d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db59e6abd91d4ebf8120160ba949b1cc" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 43441 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:52:05.468552 10388 raft_consensus.cc:1261] T fdeaf89526c74df58e8e61fc0f0fdea3 P b84095e166154f4b93e54a628dca7381 [term 1 FOLLOWER]: Refusing update from remote peer 27474c2149a74ec3a092d6f0abced2d2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:52:05.468864 10707 consensus_queue.cc:1035] T fdeaf89526c74df58e8e61fc0f0fdea3 P 27474c2149a74ec3a092d6f0abced2d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b84095e166154f4b93e54a628dca7381" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 41153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:52:05.471055 10716 mvcc.cc:205] Tried to move back new op lower bound from 7316307662710853632 to 7316307662574141440. Current Snapshot: MvccSnapshot[applied={T|T < 7316307662710853632}]
I20260808 16:52:05.471946 10721 mvcc.cc:205] Tried to move back new op lower bound from 7316307662710853632 to 7316307662574141440. Current Snapshot: MvccSnapshot[applied={T|T < 7316307662710853632}]
I20260808 16:52:05.473294 9939 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53174:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 16:52:05.473410 9939 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53174:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 16:52:05.474227 9939 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 20ca135193dc4539ac31f0fc9bb8b409: Sending DeleteTablet for 3 replicas of tablet fdeaf89526c74df58e8e61fc0f0fdea3
I20260808 16:52:05.474761 10568 tablet_service.cc:1514] Processing DeleteTablet for tablet fdeaf89526c74df58e8e61fc0f0fdea3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:52:05 UTC) from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:48676
I20260808 16:52:05.474854 9939 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53174:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 16:52:05.474983 9939 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 16:52:05.475999 10142 tablet_service.cc:1467] Processing CreateTablet for tablet aa850cd81afe4299b2ca49658cd27900 (DEFAULT_TABLE table=test-table [id=7eeb7cf9bf3f425f86a4ee5a30fa415f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:52:05.476115 10142 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aa850cd81afe4299b2ca49658cd27900. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:52:05.476214 10356 tablet_service.cc:1467] Processing CreateTablet for tablet aa850cd81afe4299b2ca49658cd27900 (DEFAULT_TABLE table=test-table [id=7eeb7cf9bf3f425f86a4ee5a30fa415f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:52:05.476291 10357 tablet_service.cc:1514] Processing DeleteTablet for tablet fdeaf89526c74df58e8e61fc0f0fdea3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:52:05 UTC) from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:35388
I20260808 16:52:05.476344 10356 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aa850cd81afe4299b2ca49658cd27900. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:52:05.476606 10143 tablet_service.cc:1514] Processing DeleteTablet for tablet fdeaf89526c74df58e8e61fc0f0fdea3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:52:05 UTC) from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:35332
I20260808 16:52:05.477028 10728 tablet_replica.cc:330] T fdeaf89526c74df58e8e61fc0f0fdea3 P b84095e166154f4b93e54a628dca7381: stopping tablet replica
I20260808 16:52:05.477159 10729 tablet_replica.cc:330] T fdeaf89526c74df58e8e61fc0f0fdea3 P db59e6abd91d4ebf8120160ba949b1cc: stopping tablet replica
I20260808 16:52:05.477294 10729 raft_consensus.cc:2229] T fdeaf89526c74df58e8e61fc0f0fdea3 P db59e6abd91d4ebf8120160ba949b1cc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 16:52:05.477450 10728 raft_consensus.cc:2229] T fdeaf89526c74df58e8e61fc0f0fdea3 P b84095e166154f4b93e54a628dca7381 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 16:52:05.477600 10728 raft_consensus.cc:2258] T fdeaf89526c74df58e8e61fc0f0fdea3 P b84095e166154f4b93e54a628dca7381 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:52:05.477474 10700 tablet_bootstrap.cc:492] T aa850cd81afe4299b2ca49658cd27900 P db59e6abd91d4ebf8120160ba949b1cc: Bootstrap starting.
I20260808 16:52:05.477945 10568 tablet_service.cc:1467] Processing CreateTablet for tablet aa850cd81afe4299b2ca49658cd27900 (DEFAULT_TABLE table=test-table [id=7eeb7cf9bf3f425f86a4ee5a30fa415f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:52:05.478034 10568 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aa850cd81afe4299b2ca49658cd27900. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:52:05.478056 10700 tablet_bootstrap.cc:654] T aa850cd81afe4299b2ca49658cd27900 P db59e6abd91d4ebf8120160ba949b1cc: Neither blocks nor log segments found. Creating new log.
I20260808 16:52:05.478561 10729 raft_consensus.cc:2258] T fdeaf89526c74df58e8e61fc0f0fdea3 P db59e6abd91d4ebf8120160ba949b1cc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:52:05.477730 10701 tablet_bootstrap.cc:492] T aa850cd81afe4299b2ca49658cd27900 P b84095e166154f4b93e54a628dca7381: Bootstrap starting.
I20260808 16:52:05.478992 10701 tablet_bootstrap.cc:654] T aa850cd81afe4299b2ca49658cd27900 P b84095e166154f4b93e54a628dca7381: Neither blocks nor log segments found. Creating new log.
I20260808 16:52:05.479421 10699 tablet_bootstrap.cc:492] T aa850cd81afe4299b2ca49658cd27900 P 27474c2149a74ec3a092d6f0abced2d2: Bootstrap starting.
I20260808 16:52:05.479463 10700 tablet_bootstrap.cc:492] T aa850cd81afe4299b2ca49658cd27900 P db59e6abd91d4ebf8120160ba949b1cc: No bootstrap required, opened a new log
I20260808 16:52:05.479638 10701 tablet_bootstrap.cc:492] T aa850cd81afe4299b2ca49658cd27900 P b84095e166154f4b93e54a628dca7381: No bootstrap required, opened a new log
I20260808 16:52:05.479637 10700 ts_tablet_manager.cc:1397] T aa850cd81afe4299b2ca49658cd27900 P db59e6abd91d4ebf8120160ba949b1cc: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 16:52:05.479681 10701 ts_tablet_manager.cc:1397] T aa850cd81afe4299b2ca49658cd27900 P b84095e166154f4b93e54a628dca7381: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 16:52:05.479775 10700 raft_consensus.cc:348] T aa850cd81afe4299b2ca49658cd27900 P db59e6abd91d4ebf8120160ba949b1cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db59e6abd91d4ebf8120160ba949b1cc" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 43441 } } peers { permanent_uuid: "b84095e166154f4b93e54a628dca7381" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 41153 } } peers { permanent_uuid: "27474c2149a74ec3a092d6f0abced2d2" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 36397 } }
I20260808 16:52:05.479799 10699 tablet_bootstrap.cc:654] T aa850cd81afe4299b2ca49658cd27900 P 27474c2149a74ec3a092d6f0abced2d2: Neither blocks nor log segments found. Creating new log.
I20260808 16:52:05.479820 10700 raft_consensus.cc:374] T aa850cd81afe4299b2ca49658cd27900 P db59e6abd91d4ebf8120160ba949b1cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:52:05.479831 10700 raft_consensus.cc:729] T aa850cd81afe4299b2ca49658cd27900 P db59e6abd91d4ebf8120160ba949b1cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db59e6abd91d4ebf8120160ba949b1cc, State: Initialized, Role: FOLLOWER
I20260808 16:52:05.479810 10701 raft_consensus.cc:348] T aa850cd81afe4299b2ca49658cd27900 P b84095e166154f4b93e54a628dca7381 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db59e6abd91d4ebf8120160ba949b1cc" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 43441 } } peers { permanent_uuid: "b84095e166154f4b93e54a628dca7381" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 41153 } } peers { permanent_uuid: "27474c2149a74ec3a092d6f0abced2d2" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 36397 } }
I20260808 16:52:05.479871 10701 raft_consensus.cc:374] T aa850cd81afe4299b2ca49658cd27900 P b84095e166154f4b93e54a628dca7381 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:52:05.479866 10700 consensus_queue.cc:260] T aa850cd81afe4299b2ca49658cd27900 P db59e6abd91d4ebf8120160ba949b1cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db59e6abd91d4ebf8120160ba949b1cc" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 43441 } } peers { permanent_uuid: "b84095e166154f4b93e54a628dca7381" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 41153 } } peers { permanent_uuid: "27474c2149a74ec3a092d6f0abced2d2" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 36397 } }
I20260808 16:52:05.479889 10701 raft_consensus.cc:729] T aa850cd81afe4299b2ca49658cd27900 P b84095e166154f4b93e54a628dca7381 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b84095e166154f4b93e54a628dca7381, State: Initialized, Role: FOLLOWER
I20260808 16:52:05.479938 10700 ts_tablet_manager.cc:1428] T aa850cd81afe4299b2ca49658cd27900 P db59e6abd91d4ebf8120160ba949b1cc: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:05.479931 10701 consensus_queue.cc:260] T aa850cd81afe4299b2ca49658cd27900 P b84095e166154f4b93e54a628dca7381 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db59e6abd91d4ebf8120160ba949b1cc" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 43441 } } peers { permanent_uuid: "b84095e166154f4b93e54a628dca7381" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 41153 } } peers { permanent_uuid: "27474c2149a74ec3a092d6f0abced2d2" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 36397 } }
I20260808 16:52:05.480003 10701 ts_tablet_manager.cc:1428] T aa850cd81afe4299b2ca49658cd27900 P b84095e166154f4b93e54a628dca7381: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:05.480733 10729 ts_tablet_manager.cc:1905] T fdeaf89526c74df58e8e61fc0f0fdea3 P db59e6abd91d4ebf8120160ba949b1cc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:52:05.480990 10728 ts_tablet_manager.cc:1905] T fdeaf89526c74df58e8e61fc0f0fdea3 P b84095e166154f4b93e54a628dca7381: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:52:05.481185 10699 tablet_bootstrap.cc:492] T aa850cd81afe4299b2ca49658cd27900 P 27474c2149a74ec3a092d6f0abced2d2: No bootstrap required, opened a new log
I20260808 16:52:05.481241 10699 ts_tablet_manager.cc:1397] T aa850cd81afe4299b2ca49658cd27900 P 27474c2149a74ec3a092d6f0abced2d2: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 16:52:05.481380 10699 raft_consensus.cc:348] T aa850cd81afe4299b2ca49658cd27900 P 27474c2149a74ec3a092d6f0abced2d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db59e6abd91d4ebf8120160ba949b1cc" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 43441 } } peers { permanent_uuid: "b84095e166154f4b93e54a628dca7381" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 41153 } } peers { permanent_uuid: "27474c2149a74ec3a092d6f0abced2d2" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 36397 } }
I20260808 16:52:05.481446 10699 raft_consensus.cc:374] T aa850cd81afe4299b2ca49658cd27900 P 27474c2149a74ec3a092d6f0abced2d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:52:05.481465 10699 raft_consensus.cc:729] T aa850cd81afe4299b2ca49658cd27900 P 27474c2149a74ec3a092d6f0abced2d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27474c2149a74ec3a092d6f0abced2d2, State: Initialized, Role: FOLLOWER
I20260808 16:52:05.481508 10699 consensus_queue.cc:260] T aa850cd81afe4299b2ca49658cd27900 P 27474c2149a74ec3a092d6f0abced2d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db59e6abd91d4ebf8120160ba949b1cc" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 43441 } } peers { permanent_uuid: "b84095e166154f4b93e54a628dca7381" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 41153 } } peers { permanent_uuid: "27474c2149a74ec3a092d6f0abced2d2" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 36397 } }
I20260808 16:52:05.481601 10699 ts_tablet_manager.cc:1428] T aa850cd81afe4299b2ca49658cd27900 P 27474c2149a74ec3a092d6f0abced2d2: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
W20260808 16:52:05.480999 10240 tablet.cc:2369] T aa850cd81afe4299b2ca49658cd27900 P db59e6abd91d4ebf8120160ba949b1cc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 16:52:05.482373 10728 ts_tablet_manager.cc:1918] T fdeaf89526c74df58e8e61fc0f0fdea3 P b84095e166154f4b93e54a628dca7381: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 16:52:05.482497 10728 log.cc:1196] T fdeaf89526c74df58e8e61fc0f0fdea3 P b84095e166154f4b93e54a628dca7381: Deleting WAL directory at /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786207919384338-8288-0/minicluster-data/ts-1/wal/wals/fdeaf89526c74df58e8e61fc0f0fdea3
I20260808 16:52:05.482811 10728 ts_tablet_manager.cc:1939] T fdeaf89526c74df58e8e61fc0f0fdea3 P b84095e166154f4b93e54a628dca7381: Deleting consensus metadata
I20260808 16:52:05.483178 10705 raft_consensus.cc:482] T aa850cd81afe4299b2ca49658cd27900 P db59e6abd91d4ebf8120160ba949b1cc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 16:52:05.483232 10705 raft_consensus.cc:504] T aa850cd81afe4299b2ca49658cd27900 P db59e6abd91d4ebf8120160ba949b1cc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db59e6abd91d4ebf8120160ba949b1cc" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 43441 } } peers { permanent_uuid: "b84095e166154f4b93e54a628dca7381" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 41153 } } peers { permanent_uuid: "27474c2149a74ec3a092d6f0abced2d2" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 36397 } }
I20260808 16:52:05.483206 9921 catalog_manager.cc:4841] TS b84095e166154f4b93e54a628dca7381 (127.8.24.2:41153): tablet fdeaf89526c74df58e8e61fc0f0fdea3 (table test-table [id=be152bf962c74ab1a03ae0bc0d20a980]) successfully deleted
I20260808 16:52:05.483458 10705 leader_election.cc:288] T aa850cd81afe4299b2ca49658cd27900 P db59e6abd91d4ebf8120160ba949b1cc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b84095e166154f4b93e54a628dca7381 (127.8.24.2:41153), 27474c2149a74ec3a092d6f0abced2d2 (127.8.24.3:36397)
I20260808 16:52:05.483630 10729 ts_tablet_manager.cc:1918] T fdeaf89526c74df58e8e61fc0f0fdea3 P db59e6abd91d4ebf8120160ba949b1cc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 16:52:05.483675 10729 log.cc:1196] T fdeaf89526c74df58e8e61fc0f0fdea3 P db59e6abd91d4ebf8120160ba949b1cc: Deleting WAL directory at /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786207919384338-8288-0/minicluster-data/ts-0/wal/wals/fdeaf89526c74df58e8e61fc0f0fdea3
I20260808 16:52:05.484022 10729 ts_tablet_manager.cc:1939] T fdeaf89526c74df58e8e61fc0f0fdea3 P db59e6abd91d4ebf8120160ba949b1cc: Deleting consensus metadata
I20260808 16:52:05.484361 9919 catalog_manager.cc:4841] TS db59e6abd91d4ebf8120160ba949b1cc (127.8.24.1:43441): tablet fdeaf89526c74df58e8e61fc0f0fdea3 (table test-table [id=be152bf962c74ab1a03ae0bc0d20a980]) successfully deleted
I20260808 16:52:05.484458 10727 tablet_replica.cc:330] T fdeaf89526c74df58e8e61fc0f0fdea3 P 27474c2149a74ec3a092d6f0abced2d2: stopping tablet replica
I20260808 16:52:05.484558 10727 raft_consensus.cc:2229] T fdeaf89526c74df58e8e61fc0f0fdea3 P 27474c2149a74ec3a092d6f0abced2d2 [term 1 LEADER]: Raft consensus shutting down.
I20260808 16:52:05.484637 10727 raft_consensus.cc:2258] T fdeaf89526c74df58e8e61fc0f0fdea3 P 27474c2149a74ec3a092d6f0abced2d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:52:05.486595 10727 ts_tablet_manager.cc:1905] T fdeaf89526c74df58e8e61fc0f0fdea3 P 27474c2149a74ec3a092d6f0abced2d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:52:05.487201 10695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:05.483890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:53183 (local address 127.8.24.2:41153)
0808 16:52:05.483922 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:05.483924 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:05.483931 (+ 7us) server_negotiation.cc:407] Connection header received
0808 16:52:05.483951 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:05.483952 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:05.483959 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:05.484016 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:52:05.484169 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:05.484801 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:05.486318 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:05.486826 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:05.486894 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:05.486904 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:05.487106 (+ 202us) server_negotiation.cc:299] Negotiation successful
0808 16:52:05.487143 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260808 16:52:05.487432 10735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:05.483641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.2:41153 (local address 127.8.24.1:53183)
0808 16:52:05.483756 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:05.483766 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:05.483818 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:05.484034 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:05.484034 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:05.484037 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:52:05.484077 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:05.484079 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:05.485236 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:05.485237 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:05.486245 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:05.486248 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:05.486952 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:05.486953 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:05.487024 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:05.487028 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 16:52:05.487316 (+ 288us) client_negotiation.cc:240] Negotiation successful
0808 16:52:05.487358 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":46,"threads_started":1}
I20260808 16:52:05.487633 10388 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa850cd81afe4299b2ca49658cd27900" candidate_uuid: "db59e6abd91d4ebf8120160ba949b1cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b84095e166154f4b93e54a628dca7381" is_pre_election: true
I20260808 16:52:05.487694 10388 raft_consensus.cc:2454] T aa850cd81afe4299b2ca49658cd27900 P b84095e166154f4b93e54a628dca7381 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db59e6abd91d4ebf8120160ba949b1cc in term 0.
I20260808 16:52:05.487869 10076 leader_election.cc:302] T aa850cd81afe4299b2ca49658cd27900 P db59e6abd91d4ebf8120160ba949b1cc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b84095e166154f4b93e54a628dca7381, db59e6abd91d4ebf8120160ba949b1cc; no voters:
I20260808 16:52:05.487993 10705 raft_consensus.cc:2781] T aa850cd81afe4299b2ca49658cd27900 P db59e6abd91d4ebf8120160ba949b1cc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 16:52:05.488016 10727 ts_tablet_manager.cc:1918] T fdeaf89526c74df58e8e61fc0f0fdea3 P 27474c2149a74ec3a092d6f0abced2d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 16:52:05.488049 10705 raft_consensus.cc:482] T aa850cd81afe4299b2ca49658cd27900 P db59e6abd91d4ebf8120160ba949b1cc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 16:52:05.488070 10705 raft_consensus.cc:3037] T aa850cd81afe4299b2ca49658cd27900 P db59e6abd91d4ebf8120160ba949b1cc [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:52:05.488073 10727 log.cc:1196] T fdeaf89526c74df58e8e61fc0f0fdea3 P 27474c2149a74ec3a092d6f0abced2d2: Deleting WAL directory at /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786207919384338-8288-0/minicluster-data/ts-2/wal/wals/fdeaf89526c74df58e8e61fc0f0fdea3
I20260808 16:52:05.488281 10727 ts_tablet_manager.cc:1939] T fdeaf89526c74df58e8e61fc0f0fdea3 P 27474c2149a74ec3a092d6f0abced2d2: Deleting consensus metadata
I20260808 16:52:05.488662 9921 catalog_manager.cc:4841] TS 27474c2149a74ec3a092d6f0abced2d2 (127.8.24.3:36397): tablet fdeaf89526c74df58e8e61fc0f0fdea3 (table test-table [id=be152bf962c74ab1a03ae0bc0d20a980]) successfully deleted
I20260808 16:52:05.488766 10705 raft_consensus.cc:504] T aa850cd81afe4299b2ca49658cd27900 P db59e6abd91d4ebf8120160ba949b1cc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db59e6abd91d4ebf8120160ba949b1cc" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 43441 } } peers { permanent_uuid: "b84095e166154f4b93e54a628dca7381" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 41153 } } peers { permanent_uuid: "27474c2149a74ec3a092d6f0abced2d2" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 36397 } }
I20260808 16:52:05.488885 10705 leader_election.cc:288] T aa850cd81afe4299b2ca49658cd27900 P db59e6abd91d4ebf8120160ba949b1cc [CANDIDATE]: Term 1 election: Requested vote from peers b84095e166154f4b93e54a628dca7381 (127.8.24.2:41153), 27474c2149a74ec3a092d6f0abced2d2 (127.8.24.3:36397)
I20260808 16:52:05.489019 10388 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa850cd81afe4299b2ca49658cd27900" candidate_uuid: "db59e6abd91d4ebf8120160ba949b1cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b84095e166154f4b93e54a628dca7381"
I20260808 16:52:05.489084 10388 raft_consensus.cc:3037] T aa850cd81afe4299b2ca49658cd27900 P b84095e166154f4b93e54a628dca7381 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:52:05.489234 10697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:05.483832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:55839 (local address 127.8.24.3:36397)
0808 16:52:05.483878 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:05.483880 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:05.484249 (+ 369us) server_negotiation.cc:407] Connection header received
0808 16:52:05.484339 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:05.484340 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:05.484360 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:05.484388 (+ 28us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:52:05.484760 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:05.485398 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:05.487068 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:05.487613 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:05.487659 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:05.487668 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:05.489139 (+ 1471us) server_negotiation.cc:299] Negotiation successful
0808 16:52:05.489176 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 16:52:05.489451 10736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:05.483777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.3:36397 (local address 127.8.24.1:55839)
0808 16:52:05.484192 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:05.484202 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:05.484274 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:05.484484 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:05.484485 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:05.484487 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:52:05.484636 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:05.484638 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:05.485458 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:05.485459 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:05.486972 (+ 1513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:05.486975 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:05.488973 (+ 1998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:05.488974 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:05.489051 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:05.489055 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 16:52:05.489335 (+ 280us) client_negotiation.cc:240] Negotiation successful
0808 16:52:05.489378 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":40,"threads_started":1}
I20260808 16:52:05.489701 10388 raft_consensus.cc:2454] T aa850cd81afe4299b2ca49658cd27900 P b84095e166154f4b93e54a628dca7381 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db59e6abd91d4ebf8120160ba949b1cc in term 1.
I20260808 16:52:05.489787 10599 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa850cd81afe4299b2ca49658cd27900" candidate_uuid: "db59e6abd91d4ebf8120160ba949b1cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27474c2149a74ec3a092d6f0abced2d2" is_pre_election: true
I20260808 16:52:05.489804 10598 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa850cd81afe4299b2ca49658cd27900" candidate_uuid: "db59e6abd91d4ebf8120160ba949b1cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27474c2149a74ec3a092d6f0abced2d2"
I20260808 16:52:05.489874 10076 leader_election.cc:302] T aa850cd81afe4299b2ca49658cd27900 P db59e6abd91d4ebf8120160ba949b1cc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b84095e166154f4b93e54a628dca7381, db59e6abd91d4ebf8120160ba949b1cc; no voters:
I20260808 16:52:05.489892 10599 raft_consensus.cc:2454] T aa850cd81afe4299b2ca49658cd27900 P 27474c2149a74ec3a092d6f0abced2d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db59e6abd91d4ebf8120160ba949b1cc in term 0.
I20260808 16:52:05.489943 10705 raft_consensus.cc:2781] T aa850cd81afe4299b2ca49658cd27900 P db59e6abd91d4ebf8120160ba949b1cc [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:52:05.490018 10705 raft_consensus.cc:686] T aa850cd81afe4299b2ca49658cd27900 P db59e6abd91d4ebf8120160ba949b1cc [term 1 LEADER]: Becoming Leader. State: Replica: db59e6abd91d4ebf8120160ba949b1cc, State: Running, Role: LEADER
I20260808 16:52:05.490088 10705 consensus_queue.cc:237] T aa850cd81afe4299b2ca49658cd27900 P db59e6abd91d4ebf8120160ba949b1cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db59e6abd91d4ebf8120160ba949b1cc" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 43441 } } peers { permanent_uuid: "b84095e166154f4b93e54a628dca7381" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 41153 } } peers { permanent_uuid: "27474c2149a74ec3a092d6f0abced2d2" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 36397 } }
I20260808 16:52:05.490612 9933 catalog_manager.cc:5495] T aa850cd81afe4299b2ca49658cd27900 P db59e6abd91d4ebf8120160ba949b1cc reported cstate change: term changed from 0 to 1, leader changed from <none> to db59e6abd91d4ebf8120160ba949b1cc (127.8.24.1). New cstate: current_term: 1 leader_uuid: "db59e6abd91d4ebf8120160ba949b1cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db59e6abd91d4ebf8120160ba949b1cc" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 43441 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b84095e166154f4b93e54a628dca7381" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 41153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "27474c2149a74ec3a092d6f0abced2d2" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 36397 } health_report { overall_health: UNKNOWN } } }
Aug 08 16:52:05 dist-test-slave-6zbq krb5kdc[9872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207924, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260808 16:52:05.521778 10749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:05.514939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:38409 (local address 127.0.0.1:53182)
0808 16:52:05.515090 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:05.515101 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:05.515168 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:05.515284 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:05.515284 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:05.515663 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:05.515713 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:05.515716 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:05.516375 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:05.516376 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:05.517198 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:05.517202 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:05.517542 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:05.517543 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:05.517601 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:05.518073 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:05.518078 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:05.519507 (+ 1429us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:52:05.520836 (+ 1329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:05.520843 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:05.520845 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:05.521018 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:05.521196 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:05.521197 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:05.521197 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:05.521257 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:05.521451 (+ 194us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:52:05.521458 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:52:05.521503 (+ 45us) client_negotiation.cc:769] Sending connection context
0808 16:52:05.521561 (+ 58us) client_negotiation.cc:240] Negotiation successful
0808 16:52:05.521677 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":81,"threads_started":1}
I20260808 16:52:05.521834 10007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:05.515000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53182 (local address 127.8.24.62:38409)
0808 16:52:05.515054 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:05.515056 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:05.515158 (+ 102us) server_negotiation.cc:407] Connection header received
0808 16:52:05.515231 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:05.515232 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:05.515240 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:05.515275 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:05.515800 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:05.516265 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:05.517275 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:05.517481 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:05.517533 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:05.519582 (+ 2049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:05.519583 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:05.519584 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:05.519590 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:05.520764 (+ 1174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:05.521078 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:05.521080 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:05.521080 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:05.521142 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:05.521304 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:05.521305 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:05.521305 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:05.521398 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:05.521451 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:05.521589 (+ 138us) server_negotiation.cc:299] Negotiation successful
0808 16:52:05.521736 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":84,"server-negotiator.queue_time_us":30}
I20260808 16:52:05.523785 9933 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:53182:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 16:52:05.524021 9933 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 16:52:05.525257 10356 tablet_service.cc:1467] Processing CreateTablet for tablet 6355a47f16cc4be59a2f510bc7604ff3 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=0e2e920b7a6e40a2b0675513f05aa987]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 16:52:05.525439 10356 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6355a47f16cc4be59a2f510bc7604ff3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:52:05.526525 10568 tablet_service.cc:1467] Processing CreateTablet for tablet 6355a47f16cc4be59a2f510bc7604ff3 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=0e2e920b7a6e40a2b0675513f05aa987]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 16:52:05.526635 10568 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6355a47f16cc4be59a2f510bc7604ff3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:52:05.525275 10142 tablet_service.cc:1467] Processing CreateTablet for tablet 6355a47f16cc4be59a2f510bc7604ff3 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=0e2e920b7a6e40a2b0675513f05aa987]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 16:52:05.527818 10142 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6355a47f16cc4be59a2f510bc7604ff3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:52:05.529191 10700 tablet_bootstrap.cc:492] T 6355a47f16cc4be59a2f510bc7604ff3 P db59e6abd91d4ebf8120160ba949b1cc: Bootstrap starting.
I20260808 16:52:05.529264 10701 tablet_bootstrap.cc:492] T 6355a47f16cc4be59a2f510bc7604ff3 P b84095e166154f4b93e54a628dca7381: Bootstrap starting.
I20260808 16:52:05.529608 10700 tablet_bootstrap.cc:654] T 6355a47f16cc4be59a2f510bc7604ff3 P db59e6abd91d4ebf8120160ba949b1cc: Neither blocks nor log segments found. Creating new log.
I20260808 16:52:05.529685 10701 tablet_bootstrap.cc:654] T 6355a47f16cc4be59a2f510bc7604ff3 P b84095e166154f4b93e54a628dca7381: Neither blocks nor log segments found. Creating new log.
I20260808 16:52:05.529997 10699 tablet_bootstrap.cc:492] T 6355a47f16cc4be59a2f510bc7604ff3 P 27474c2149a74ec3a092d6f0abced2d2: Bootstrap starting.
I20260808 16:52:05.530095 10700 tablet_bootstrap.cc:492] T 6355a47f16cc4be59a2f510bc7604ff3 P db59e6abd91d4ebf8120160ba949b1cc: No bootstrap required, opened a new log
I20260808 16:52:05.530141 10700 ts_tablet_manager.cc:1397] T 6355a47f16cc4be59a2f510bc7604ff3 P db59e6abd91d4ebf8120160ba949b1cc: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260808 16:52:05.530241 10700 raft_consensus.cc:348] T 6355a47f16cc4be59a2f510bc7604ff3 P db59e6abd91d4ebf8120160ba949b1cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db59e6abd91d4ebf8120160ba949b1cc" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 43441 } } peers { permanent_uuid: "b84095e166154f4b93e54a628dca7381" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 41153 } } peers { permanent_uuid: "27474c2149a74ec3a092d6f0abced2d2" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 36397 } }
I20260808 16:52:05.530280 10700 raft_consensus.cc:374] T 6355a47f16cc4be59a2f510bc7604ff3 P db59e6abd91d4ebf8120160ba949b1cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:52:05.530293 10700 raft_consensus.cc:729] T 6355a47f16cc4be59a2f510bc7604ff3 P db59e6abd91d4ebf8120160ba949b1cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db59e6abd91d4ebf8120160ba949b1cc, State: Initialized, Role: FOLLOWER
I20260808 16:52:05.530311 10699 tablet_bootstrap.cc:654] T 6355a47f16cc4be59a2f510bc7604ff3 P 27474c2149a74ec3a092d6f0abced2d2: Neither blocks nor log segments found. Creating new log.
I20260808 16:52:05.530326 10700 consensus_queue.cc:260] T 6355a47f16cc4be59a2f510bc7604ff3 P db59e6abd91d4ebf8120160ba949b1cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db59e6abd91d4ebf8120160ba949b1cc" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 43441 } } peers { permanent_uuid: "b84095e166154f4b93e54a628dca7381" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 41153 } } peers { permanent_uuid: "27474c2149a74ec3a092d6f0abced2d2" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 36397 } }
I20260808 16:52:05.530428 10700 ts_tablet_manager.cc:1428] T 6355a47f16cc4be59a2f510bc7604ff3 P db59e6abd91d4ebf8120160ba949b1cc: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:05.530471 10705 tablet_replica.cc:439] T 6355a47f16cc4be59a2f510bc7604ff3 P db59e6abd91d4ebf8120160ba949b1cc: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db59e6abd91d4ebf8120160ba949b1cc" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 43441 } } peers { permanent_uuid: "b84095e166154f4b93e54a628dca7381" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 41153 } } peers { permanent_uuid: "27474c2149a74ec3a092d6f0abced2d2" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 36397 } } }
I20260808 16:52:05.530532 10705 tablet_replica.cc:442] T 6355a47f16cc4be59a2f510bc7604ff3 P db59e6abd91d4ebf8120160ba949b1cc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 16:52:05.530756 10701 tablet_bootstrap.cc:492] T 6355a47f16cc4be59a2f510bc7604ff3 P b84095e166154f4b93e54a628dca7381: No bootstrap required, opened a new log
I20260808 16:52:05.530795 10701 ts_tablet_manager.cc:1397] T 6355a47f16cc4be59a2f510bc7604ff3 P b84095e166154f4b93e54a628dca7381: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 16:52:05.530902 10701 raft_consensus.cc:348] T 6355a47f16cc4be59a2f510bc7604ff3 P b84095e166154f4b93e54a628dca7381 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db59e6abd91d4ebf8120160ba949b1cc" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 43441 } } peers { permanent_uuid: "b84095e166154f4b93e54a628dca7381" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 41153 } } peers { permanent_uuid: "27474c2149a74ec3a092d6f0abced2d2" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 36397 } }
I20260808 16:52:05.530953 10701 raft_consensus.cc:374] T 6355a47f16cc4be59a2f510bc7604ff3 P b84095e166154f4b93e54a628dca7381 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:52:05.530972 10701 raft_consensus.cc:729] T 6355a47f16cc4be59a2f510bc7604ff3 P b84095e166154f4b93e54a628dca7381 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b84095e166154f4b93e54a628dca7381, State: Initialized, Role: FOLLOWER
I20260808 16:52:05.531004 10701 consensus_queue.cc:260] T 6355a47f16cc4be59a2f510bc7604ff3 P b84095e166154f4b93e54a628dca7381 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db59e6abd91d4ebf8120160ba949b1cc" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 43441 } } peers { permanent_uuid: "b84095e166154f4b93e54a628dca7381" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 41153 } } peers { permanent_uuid: "27474c2149a74ec3a092d6f0abced2d2" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 36397 } }
I20260808 16:52:05.531096 10701 ts_tablet_manager.cc:1428] T 6355a47f16cc4be59a2f510bc7604ff3 P b84095e166154f4b93e54a628dca7381: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:05.532012 10706 tablet_replica.cc:439] T 6355a47f16cc4be59a2f510bc7604ff3 P b84095e166154f4b93e54a628dca7381: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db59e6abd91d4ebf8120160ba949b1cc" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 43441 } } peers { permanent_uuid: "b84095e166154f4b93e54a628dca7381" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 41153 } } peers { permanent_uuid: "27474c2149a74ec3a092d6f0abced2d2" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 36397 } } }
I20260808 16:52:05.532114 10706 tablet_replica.cc:442] T 6355a47f16cc4be59a2f510bc7604ff3 P b84095e166154f4b93e54a628dca7381: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 16:52:05.532589 10699 tablet_bootstrap.cc:492] T 6355a47f16cc4be59a2f510bc7604ff3 P 27474c2149a74ec3a092d6f0abced2d2: No bootstrap required, opened a new log
I20260808 16:52:05.532634 10699 ts_tablet_manager.cc:1397] T 6355a47f16cc4be59a2f510bc7604ff3 P 27474c2149a74ec3a092d6f0abced2d2: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260808 16:52:05.532764 10699 raft_consensus.cc:348] T 6355a47f16cc4be59a2f510bc7604ff3 P 27474c2149a74ec3a092d6f0abced2d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db59e6abd91d4ebf8120160ba949b1cc" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 43441 } } peers { permanent_uuid: "b84095e166154f4b93e54a628dca7381" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 41153 } } peers { permanent_uuid: "27474c2149a74ec3a092d6f0abced2d2" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 36397 } }
I20260808 16:52:05.532815 10699 raft_consensus.cc:374] T 6355a47f16cc4be59a2f510bc7604ff3 P 27474c2149a74ec3a092d6f0abced2d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:52:05.532832 10699 raft_consensus.cc:729] T 6355a47f16cc4be59a2f510bc7604ff3 P 27474c2149a74ec3a092d6f0abced2d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27474c2149a74ec3a092d6f0abced2d2, State: Initialized, Role: FOLLOWER
I20260808 16:52:05.532868 10699 consensus_queue.cc:260] T 6355a47f16cc4be59a2f510bc7604ff3 P 27474c2149a74ec3a092d6f0abced2d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db59e6abd91d4ebf8120160ba949b1cc" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 43441 } } peers { permanent_uuid: "b84095e166154f4b93e54a628dca7381" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 41153 } } peers { permanent_uuid: "27474c2149a74ec3a092d6f0abced2d2" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 36397 } }
I20260808 16:52:05.532997 10699 ts_tablet_manager.cc:1428] T 6355a47f16cc4be59a2f510bc7604ff3 P 27474c2149a74ec3a092d6f0abced2d2: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:05.532981 10720 tablet_replica.cc:439] T 6355a47f16cc4be59a2f510bc7604ff3 P 27474c2149a74ec3a092d6f0abced2d2: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db59e6abd91d4ebf8120160ba949b1cc" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 43441 } } peers { permanent_uuid: "b84095e166154f4b93e54a628dca7381" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 41153 } } peers { permanent_uuid: "27474c2149a74ec3a092d6f0abced2d2" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 36397 } } }
I20260808 16:52:05.533083 10720 tablet_replica.cc:442] T 6355a47f16cc4be59a2f510bc7604ff3 P 27474c2149a74ec3a092d6f0abced2d2: This TxnStatusTablet replica's current role is: FOLLOWER
W20260808 16:52:05.537249 10671 tablet.cc:2369] T 6355a47f16cc4be59a2f510bc7604ff3 P 27474c2149a74ec3a092d6f0abced2d2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 16:52:05.546419 10706 raft_consensus.cc:482] T 6355a47f16cc4be59a2f510bc7604ff3 P b84095e166154f4b93e54a628dca7381 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 16:52:05.546489 10706 raft_consensus.cc:504] T 6355a47f16cc4be59a2f510bc7604ff3 P b84095e166154f4b93e54a628dca7381 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db59e6abd91d4ebf8120160ba949b1cc" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 43441 } } peers { permanent_uuid: "b84095e166154f4b93e54a628dca7381" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 41153 } } peers { permanent_uuid: "27474c2149a74ec3a092d6f0abced2d2" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 36397 } }
I20260808 16:52:05.546787 10706 leader_election.cc:288] T 6355a47f16cc4be59a2f510bc7604ff3 P b84095e166154f4b93e54a628dca7381 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers db59e6abd91d4ebf8120160ba949b1cc (127.8.24.1:43441), 27474c2149a74ec3a092d6f0abced2d2 (127.8.24.3:36397)
I20260808 16:52:05.549784 10292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:05.546889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.3:36397 (local address 127.8.24.2:50299)
0808 16:52:05.546955 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:05.546964 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:05.547001 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:05.547189 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:05.547190 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:05.547191 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:52:05.547253 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:05.547254 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:05.547895 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:05.547896 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:05.548847 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:05.548849 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:05.549543 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:05.549543 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:05.549621 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:05.549625 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 16:52:05.549677 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 16:52:05.549718 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260808 16:52:05.549796 10697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:05.547005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:50299 (local address 127.8.24.3:36397)
0808 16:52:05.547044 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:05.547046 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:05.547054 (+ 8us) server_negotiation.cc:407] Connection header received
0808 16:52:05.547075 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:05.547076 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:05.547084 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:05.547140 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:52:05.547316 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:05.547830 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:05.548942 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:05.549464 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:05.549544 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:05.549555 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:05.549699 (+ 144us) server_negotiation.cc:299] Negotiation successful
0808 16:52:05.549736 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 16:52:05.549971 10598 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6355a47f16cc4be59a2f510bc7604ff3" candidate_uuid: "b84095e166154f4b93e54a628dca7381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27474c2149a74ec3a092d6f0abced2d2" is_pre_election: true
I20260808 16:52:05.550045 10598 raft_consensus.cc:2454] T 6355a47f16cc4be59a2f510bc7604ff3 P 27474c2149a74ec3a092d6f0abced2d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b84095e166154f4b93e54a628dca7381 in term 0.
I20260808 16:52:05.550141 10698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:05.547070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:59459 (local address 127.8.24.1:43441)
0808 16:52:05.547106 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:05.547108 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:05.547194 (+ 86us) server_negotiation.cc:407] Connection header received
0808 16:52:05.547283 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:05.547284 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:05.547292 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:05.547319 (+ 27us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:52:05.547554 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:05.548222 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:05.549318 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:05.549718 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:05.549773 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:05.549781 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:05.550043 (+ 262us) server_negotiation.cc:299] Negotiation successful
0808 16:52:05.550082 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 16:52:05.550230 10286 leader_election.cc:302] T 6355a47f16cc4be59a2f510bc7604ff3 P b84095e166154f4b93e54a628dca7381 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27474c2149a74ec3a092d6f0abced2d2, b84095e166154f4b93e54a628dca7381; no voters:
I20260808 16:52:05.550333 10756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:05.546889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:43441 (local address 127.8.24.2:59459)
0808 16:52:05.547143 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:05.547151 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:05.547216 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:05.547373 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:05.547373 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:05.547375 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:52:05.547457 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:05.547458 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:05.548314 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:05.548315 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:05.549248 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:05.549250 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:05.549827 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:05.549828 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:05.549970 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:05.549975 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 16:52:05.550214 (+ 239us) client_negotiation.cc:240] Negotiation successful
0808 16:52:05.550257 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"mutex_wait_us":62,"thread_start_us":58,"threads_started":1}
I20260808 16:52:05.550383 10706 raft_consensus.cc:2781] T 6355a47f16cc4be59a2f510bc7604ff3 P b84095e166154f4b93e54a628dca7381 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 16:52:05.550444 10706 raft_consensus.cc:482] T 6355a47f16cc4be59a2f510bc7604ff3 P b84095e166154f4b93e54a628dca7381 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 16:52:05.550472 10706 raft_consensus.cc:3037] T 6355a47f16cc4be59a2f510bc7604ff3 P b84095e166154f4b93e54a628dca7381 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:52:05.551206 10706 raft_consensus.cc:504] T 6355a47f16cc4be59a2f510bc7604ff3 P b84095e166154f4b93e54a628dca7381 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db59e6abd91d4ebf8120160ba949b1cc" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 43441 } } peers { permanent_uuid: "b84095e166154f4b93e54a628dca7381" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 41153 } } peers { permanent_uuid: "27474c2149a74ec3a092d6f0abced2d2" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 36397 } }
I20260808 16:52:05.551338 10706 leader_election.cc:288] T 6355a47f16cc4be59a2f510bc7604ff3 P b84095e166154f4b93e54a628dca7381 [CANDIDATE]: Term 1 election: Requested vote from peers db59e6abd91d4ebf8120160ba949b1cc (127.8.24.1:43441), 27474c2149a74ec3a092d6f0abced2d2 (127.8.24.3:36397)
I20260808 16:52:05.551479 10172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6355a47f16cc4be59a2f510bc7604ff3" candidate_uuid: "b84095e166154f4b93e54a628dca7381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db59e6abd91d4ebf8120160ba949b1cc" is_pre_election: true
I20260808 16:52:05.551524 10171 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6355a47f16cc4be59a2f510bc7604ff3" candidate_uuid: "b84095e166154f4b93e54a628dca7381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db59e6abd91d4ebf8120160ba949b1cc"
I20260808 16:52:05.551559 10172 raft_consensus.cc:2454] T 6355a47f16cc4be59a2f510bc7604ff3 P db59e6abd91d4ebf8120160ba949b1cc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b84095e166154f4b93e54a628dca7381 in term 0.
I20260808 16:52:05.551600 10598 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6355a47f16cc4be59a2f510bc7604ff3" candidate_uuid: "b84095e166154f4b93e54a628dca7381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27474c2149a74ec3a092d6f0abced2d2"
I20260808 16:52:05.551685 10598 raft_consensus.cc:3037] T 6355a47f16cc4be59a2f510bc7604ff3 P 27474c2149a74ec3a092d6f0abced2d2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:52:05.552403 10598 raft_consensus.cc:2454] T 6355a47f16cc4be59a2f510bc7604ff3 P 27474c2149a74ec3a092d6f0abced2d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b84095e166154f4b93e54a628dca7381 in term 1.
I20260808 16:52:05.552644 10286 leader_election.cc:302] T 6355a47f16cc4be59a2f510bc7604ff3 P b84095e166154f4b93e54a628dca7381 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 27474c2149a74ec3a092d6f0abced2d2, b84095e166154f4b93e54a628dca7381; no voters: db59e6abd91d4ebf8120160ba949b1cc
I20260808 16:52:05.552731 10706 raft_consensus.cc:2781] T 6355a47f16cc4be59a2f510bc7604ff3 P b84095e166154f4b93e54a628dca7381 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:52:05.552876 10706 raft_consensus.cc:686] T 6355a47f16cc4be59a2f510bc7604ff3 P b84095e166154f4b93e54a628dca7381 [term 1 LEADER]: Becoming Leader. State: Replica: b84095e166154f4b93e54a628dca7381, State: Running, Role: LEADER
I20260808 16:52:05.552940 10706 consensus_queue.cc:237] T 6355a47f16cc4be59a2f510bc7604ff3 P b84095e166154f4b93e54a628dca7381 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db59e6abd91d4ebf8120160ba949b1cc" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 43441 } } peers { permanent_uuid: "b84095e166154f4b93e54a628dca7381" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 41153 } } peers { permanent_uuid: "27474c2149a74ec3a092d6f0abced2d2" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 36397 } }
I20260808 16:52:05.553269 10757 tablet_replica.cc:439] T 6355a47f16cc4be59a2f510bc7604ff3 P b84095e166154f4b93e54a628dca7381: TxnStatusTablet state changed. Reason: New leader b84095e166154f4b93e54a628dca7381. Latest consensus state: current_term: 1 leader_uuid: "b84095e166154f4b93e54a628dca7381" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db59e6abd91d4ebf8120160ba949b1cc" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 43441 } } peers { permanent_uuid: "b84095e166154f4b93e54a628dca7381" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 41153 } } peers { permanent_uuid: "27474c2149a74ec3a092d6f0abced2d2" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 36397 } } }
I20260808 16:52:05.553608 10757 tablet_replica.cc:442] T 6355a47f16cc4be59a2f510bc7604ff3 P b84095e166154f4b93e54a628dca7381: This TxnStatusTablet replica's current role is: LEADER
I20260808 16:52:05.553537 9939 catalog_manager.cc:5495] T 6355a47f16cc4be59a2f510bc7604ff3 P b84095e166154f4b93e54a628dca7381 reported cstate change: term changed from 0 to 1, leader changed from <none> to b84095e166154f4b93e54a628dca7381 (127.8.24.2). New cstate: current_term: 1 leader_uuid: "b84095e166154f4b93e54a628dca7381" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db59e6abd91d4ebf8120160ba949b1cc" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 43441 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b84095e166154f4b93e54a628dca7381" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 41153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "27474c2149a74ec3a092d6f0abced2d2" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 36397 } health_report { overall_health: UNKNOWN } } }
I20260808 16:52:05.553808 10759 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 16:52:05.553920 10759 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 16:52:05.566510 10749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:05.563481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.2:41153 (local address 127.0.0.1:35398)
0808 16:52:05.563537 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:05.563546 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:05.563583 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:05.563779 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:05.563780 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:05.563781 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 16:52:05.563823 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:05.563825 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:05.564630 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:05.564631 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:05.565521 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:05.565524 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:05.565974 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:05.565974 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:05.566038 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:05.566053 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 16:52:05.566319 (+ 266us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 16:52:05.566324 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 16:52:05.566389 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 16:52:05.566436 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260808 16:52:05.566670 10695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:05.563558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35398 (local address 127.8.24.2:41153)
0808 16:52:05.563597 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:05.563598 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:05.563632 (+ 34us) server_negotiation.cc:407] Connection header received
0808 16:52:05.563671 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:05.563672 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:05.563700 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:05.563763 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 16:52:05.563896 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:05.564548 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:05.565614 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:05.565894 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:05.565965 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:05.566145 (+ 180us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 16:52:05.566251 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 16:52:05.566360 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:05.566593 (+ 233us) server_negotiation.cc:299] Negotiation successful
0808 16:52:05.566613 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 16:52:05.567658 10172 raft_consensus.cc:3037] T 6355a47f16cc4be59a2f510bc7604ff3 P db59e6abd91d4ebf8120160ba949b1cc [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:52:05.567730 10598 raft_consensus.cc:1261] T 6355a47f16cc4be59a2f510bc7604ff3 P 27474c2149a74ec3a092d6f0abced2d2 [term 1 FOLLOWER]: Refusing update from remote peer b84095e166154f4b93e54a628dca7381: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:52:05.567994 10757 consensus_queue.cc:1035] T 6355a47f16cc4be59a2f510bc7604ff3 P b84095e166154f4b93e54a628dca7381 [LEADER]: Connected to new peer: Peer: permanent_uuid: "27474c2149a74ec3a092d6f0abced2d2" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 36397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:52:05.568392 10172 raft_consensus.cc:1261] T 6355a47f16cc4be59a2f510bc7604ff3 P db59e6abd91d4ebf8120160ba949b1cc [term 1 FOLLOWER]: Refusing update from remote peer b84095e166154f4b93e54a628dca7381: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:52:05.568593 10706 consensus_queue.cc:1035] T 6355a47f16cc4be59a2f510bc7604ff3 P b84095e166154f4b93e54a628dca7381 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db59e6abd91d4ebf8120160ba949b1cc" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 43441 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:52:05.568610 10720 tablet_replica.cc:439] T 6355a47f16cc4be59a2f510bc7604ff3 P 27474c2149a74ec3a092d6f0abced2d2: TxnStatusTablet state changed. Reason: New leader b84095e166154f4b93e54a628dca7381. Latest consensus state: current_term: 1 leader_uuid: "b84095e166154f4b93e54a628dca7381" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db59e6abd91d4ebf8120160ba949b1cc" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 43441 } } peers { permanent_uuid: "b84095e166154f4b93e54a628dca7381" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 41153 } } peers { permanent_uuid: "27474c2149a74ec3a092d6f0abced2d2" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 36397 } } }
I20260808 16:52:05.568696 10720 tablet_replica.cc:442] T 6355a47f16cc4be59a2f510bc7604ff3 P 27474c2149a74ec3a092d6f0abced2d2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 16:52:05.569043 10705 tablet_replica.cc:439] T 6355a47f16cc4be59a2f510bc7604ff3 P db59e6abd91d4ebf8120160ba949b1cc: TxnStatusTablet state changed. Reason: New leader b84095e166154f4b93e54a628dca7381. Latest consensus state: current_term: 1 leader_uuid: "b84095e166154f4b93e54a628dca7381" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db59e6abd91d4ebf8120160ba949b1cc" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 43441 } } peers { permanent_uuid: "b84095e166154f4b93e54a628dca7381" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 41153 } } peers { permanent_uuid: "27474c2149a74ec3a092d6f0abced2d2" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 36397 } } }
I20260808 16:52:05.569129 10705 tablet_replica.cc:442] T 6355a47f16cc4be59a2f510bc7604ff3 P db59e6abd91d4ebf8120160ba949b1cc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 16:52:05.570045 10706 tablet_replica.cc:439] T 6355a47f16cc4be59a2f510bc7604ff3 P b84095e166154f4b93e54a628dca7381: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b84095e166154f4b93e54a628dca7381" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db59e6abd91d4ebf8120160ba949b1cc" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 43441 } } peers { permanent_uuid: "b84095e166154f4b93e54a628dca7381" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 41153 } } peers { permanent_uuid: "27474c2149a74ec3a092d6f0abced2d2" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 36397 } } }
I20260808 16:52:05.570122 10706 tablet_replica.cc:442] T 6355a47f16cc4be59a2f510bc7604ff3 P b84095e166154f4b93e54a628dca7381: This TxnStatusTablet replica's current role is: LEADER
I20260808 16:52:05.570204 10706 tablet_replica.cc:439] T 6355a47f16cc4be59a2f510bc7604ff3 P b84095e166154f4b93e54a628dca7381: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b84095e166154f4b93e54a628dca7381" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db59e6abd91d4ebf8120160ba949b1cc" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 43441 } } peers { permanent_uuid: "b84095e166154f4b93e54a628dca7381" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 41153 } } peers { permanent_uuid: "27474c2149a74ec3a092d6f0abced2d2" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 36397 } } }
I20260808 16:52:05.570246 10706 tablet_replica.cc:442] T 6355a47f16cc4be59a2f510bc7604ff3 P b84095e166154f4b93e54a628dca7381: This TxnStatusTablet replica's current role is: LEADER
I20260808 16:52:05.570695 10705 tablet_replica.cc:439] T 6355a47f16cc4be59a2f510bc7604ff3 P db59e6abd91d4ebf8120160ba949b1cc: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b84095e166154f4b93e54a628dca7381" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db59e6abd91d4ebf8120160ba949b1cc" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 43441 } } peers { permanent_uuid: "b84095e166154f4b93e54a628dca7381" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 41153 } } peers { permanent_uuid: "27474c2149a74ec3a092d6f0abced2d2" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 36397 } } }
I20260808 16:52:05.570772 10705 tablet_replica.cc:442] T 6355a47f16cc4be59a2f510bc7604ff3 P db59e6abd91d4ebf8120160ba949b1cc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 16:52:05.570984 10720 tablet_replica.cc:439] T 6355a47f16cc4be59a2f510bc7604ff3 P 27474c2149a74ec3a092d6f0abced2d2: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b84095e166154f4b93e54a628dca7381" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db59e6abd91d4ebf8120160ba949b1cc" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 43441 } } peers { permanent_uuid: "b84095e166154f4b93e54a628dca7381" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 41153 } } peers { permanent_uuid: "27474c2149a74ec3a092d6f0abced2d2" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 36397 } } }
I20260808 16:52:05.571059 10720 tablet_replica.cc:442] T 6355a47f16cc4be59a2f510bc7604ff3 P 27474c2149a74ec3a092d6f0abced2d2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 16:52:05.575354 10698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:05.572299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35336 (local address 127.8.24.1:43441)
0808 16:52:05.572332 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:05.572333 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:05.572339 (+ 6us) server_negotiation.cc:407] Connection header received
0808 16:52:05.572372 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:05.572373 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:05.572389 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:05.572446 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 16:52:05.572587 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:05.573283 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:05.574335 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:05.574644 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:05.574700 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:05.574873 (+ 173us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 16:52:05.574979 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 16:52:05.575036 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:05.575257 (+ 221us) server_negotiation.cc:299] Negotiation successful
0808 16:52:05.575274 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 16:52:05.581069 10695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:05.578054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:58143 (local address 127.8.24.2:41153)
0808 16:52:05.578099 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:05.578101 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:05.578251 (+ 150us) server_negotiation.cc:407] Connection header received
0808 16:52:05.578297 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:05.578298 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:05.578306 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:05.578387 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 16:52:05.578531 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:05.579215 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:05.580273 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:05.580566 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:05.580628 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:05.580841 (+ 213us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 16:52:05.580871 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 16:52:05.580915 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:05.580981 (+ 66us) server_negotiation.cc:299] Negotiation successful
0808 16:52:05.581001 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 16:52:05.581122 10772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:05.577994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.2:41153 (local address 127.8.24.1:58143)
0808 16:52:05.578172 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:05.578183 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:05.578248 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:05.578403 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:05.578403 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:05.578404 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 16:52:05.578468 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:05.578470 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:05.579299 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:05.579300 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:05.580193 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:05.580196 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:05.580643 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:05.580645 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:05.580734 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:05.580750 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 16:52:05.580929 (+ 179us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 16:52:05.580931 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 16:52:05.580978 (+ 47us) client_negotiation.cc:240] Negotiation successful
0808 16:52:05.581021 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":73,"threads_started":1}
I20260808 16:52:05.583765 10598 raft_consensus.cc:3037] T aa850cd81afe4299b2ca49658cd27900 P 27474c2149a74ec3a092d6f0abced2d2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:52:05.583825 10388 raft_consensus.cc:1261] T aa850cd81afe4299b2ca49658cd27900 P b84095e166154f4b93e54a628dca7381 [term 1 FOLLOWER]: Refusing update from remote peer db59e6abd91d4ebf8120160ba949b1cc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:52:05.584501 10598 raft_consensus.cc:1261] T aa850cd81afe4299b2ca49658cd27900 P 27474c2149a74ec3a092d6f0abced2d2 [term 1 FOLLOWER]: Refusing update from remote peer db59e6abd91d4ebf8120160ba949b1cc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:52:05.584619 10737 consensus_queue.cc:1035] T aa850cd81afe4299b2ca49658cd27900 P db59e6abd91d4ebf8120160ba949b1cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "b84095e166154f4b93e54a628dca7381" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 41153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:52:05.584915 10737 consensus_queue.cc:1035] T aa850cd81afe4299b2ca49658cd27900 P db59e6abd91d4ebf8120160ba949b1cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "27474c2149a74ec3a092d6f0abced2d2" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 36397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:52:05.587117 10712 mvcc.cc:205] Tried to move back new op lower bound from 7316307663189635072 to 7316307662808002560. Current Snapshot: MvccSnapshot[applied={T|T < 7316307663189635072}]
I20260808 16:52:05.595297 10468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:05.591507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:43441 (local address 127.8.24.2:57655)
0808 16:52:05.591566 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:05.591577 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:05.591631 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:05.592016 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:05.592017 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:05.592018 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 16:52:05.592070 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:05.592072 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:05.593135 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:05.593136 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:05.594053 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:05.594055 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:05.594876 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:05.594877 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:05.594948 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:05.594962 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 16:52:05.595147 (+ 185us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 16:52:05.595148 (+ 1us) client_negotiation.cc:769] Sending connection context
0808 16:52:05.595185 (+ 37us) client_negotiation.cc:240] Negotiation successful
0808 16:52:05.595226 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22}
I20260808 16:52:05.595628 10698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:05.591811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:57655 (local address 127.8.24.1:43441)
0808 16:52:05.591902 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:05.591904 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:05.591911 (+ 7us) server_negotiation.cc:407] Connection header received
0808 16:52:05.591929 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:05.591930 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:05.591938 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:05.591980 (+ 42us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 16:52:05.592352 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:05.593037 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:05.594376 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:05.594664 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:05.594724 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:05.595046 (+ 322us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 16:52:05.595077 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 16:52:05.595125 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:05.595549 (+ 424us) server_negotiation.cc:299] Negotiation successful
0808 16:52:05.595568 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260808 16:52:05.604681 9933 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53174:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 16:52:05.604807 9933 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53174:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 16:52:05.605439 9933 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 20ca135193dc4539ac31f0fc9bb8b409: Sending DeleteTablet for 3 replicas of tablet aa850cd81afe4299b2ca49658cd27900
I20260808 16:52:05.605667 10142 tablet_service.cc:1514] Processing DeleteTablet for tablet aa850cd81afe4299b2ca49658cd27900 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:52:05 UTC) from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:35332
I20260808 16:52:05.605753 10568 tablet_service.cc:1514] Processing DeleteTablet for tablet aa850cd81afe4299b2ca49658cd27900 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:52:05 UTC) from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:48676
I20260808 16:52:05.605849 10727 tablet_replica.cc:330] T aa850cd81afe4299b2ca49658cd27900 P 27474c2149a74ec3a092d6f0abced2d2: stopping tablet replica
I20260808 16:52:05.605854 10729 tablet_replica.cc:330] T aa850cd81afe4299b2ca49658cd27900 P db59e6abd91d4ebf8120160ba949b1cc: stopping tablet replica
I20260808 16:52:05.605904 10727 raft_consensus.cc:2229] T aa850cd81afe4299b2ca49658cd27900 P 27474c2149a74ec3a092d6f0abced2d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 16:52:05.605903 10729 raft_consensus.cc:2229] T aa850cd81afe4299b2ca49658cd27900 P db59e6abd91d4ebf8120160ba949b1cc [term 1 LEADER]: Raft consensus shutting down.
I20260808 16:52:05.605942 10727 raft_consensus.cc:2258] T aa850cd81afe4299b2ca49658cd27900 P 27474c2149a74ec3a092d6f0abced2d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:52:05.605975 10729 raft_consensus.cc:2258] T aa850cd81afe4299b2ca49658cd27900 P db59e6abd91d4ebf8120160ba949b1cc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:52:05.606252 10727 ts_tablet_manager.cc:1905] T aa850cd81afe4299b2ca49658cd27900 P 27474c2149a74ec3a092d6f0abced2d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:52:05.606420 10729 ts_tablet_manager.cc:1905] T aa850cd81afe4299b2ca49658cd27900 P db59e6abd91d4ebf8120160ba949b1cc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:52:05.607777 10727 ts_tablet_manager.cc:1918] T aa850cd81afe4299b2ca49658cd27900 P 27474c2149a74ec3a092d6f0abced2d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 16:52:05.607842 10727 log.cc:1196] T aa850cd81afe4299b2ca49658cd27900 P 27474c2149a74ec3a092d6f0abced2d2: Deleting WAL directory at /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786207919384338-8288-0/minicluster-data/ts-2/wal/wals/aa850cd81afe4299b2ca49658cd27900
I20260808 16:52:05.607892 10729 ts_tablet_manager.cc:1918] T aa850cd81afe4299b2ca49658cd27900 P db59e6abd91d4ebf8120160ba949b1cc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 16:52:05.607944 10729 log.cc:1196] T aa850cd81afe4299b2ca49658cd27900 P db59e6abd91d4ebf8120160ba949b1cc: Deleting WAL directory at /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786207919384338-8288-0/minicluster-data/ts-0/wal/wals/aa850cd81afe4299b2ca49658cd27900
I20260808 16:52:05.608039 10727 ts_tablet_manager.cc:1939] T aa850cd81afe4299b2ca49658cd27900 P 27474c2149a74ec3a092d6f0abced2d2: Deleting consensus metadata
I20260808 16:52:05.608147 10729 ts_tablet_manager.cc:1939] T aa850cd81afe4299b2ca49658cd27900 P db59e6abd91d4ebf8120160ba949b1cc: Deleting consensus metadata
I20260808 16:52:05.608428 9919 catalog_manager.cc:4841] TS db59e6abd91d4ebf8120160ba949b1cc (127.8.24.1:43441): tablet aa850cd81afe4299b2ca49658cd27900 (table test-table [id=7eeb7cf9bf3f425f86a4ee5a30fa415f]) successfully deleted
I20260808 16:52:05.608444 10356 tablet_service.cc:1514] Processing DeleteTablet for tablet aa850cd81afe4299b2ca49658cd27900 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:52:05 UTC) from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:35388
I20260808 16:52:05.608512 9921 catalog_manager.cc:4841] TS 27474c2149a74ec3a092d6f0abced2d2 (127.8.24.3:36397): tablet aa850cd81afe4299b2ca49658cd27900 (table test-table [id=7eeb7cf9bf3f425f86a4ee5a30fa415f]) successfully deleted
I20260808 16:52:05.608521 10728 tablet_replica.cc:330] T aa850cd81afe4299b2ca49658cd27900 P b84095e166154f4b93e54a628dca7381: stopping tablet replica
I20260808 16:52:05.608565 10728 raft_consensus.cc:2229] T aa850cd81afe4299b2ca49658cd27900 P b84095e166154f4b93e54a628dca7381 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 16:52:05.608601 10728 raft_consensus.cc:2258] T aa850cd81afe4299b2ca49658cd27900 P b84095e166154f4b93e54a628dca7381 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:52:05.608840 10728 ts_tablet_manager.cc:1905] T aa850cd81afe4299b2ca49658cd27900 P b84095e166154f4b93e54a628dca7381: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:52:05.610368 10728 ts_tablet_manager.cc:1918] T aa850cd81afe4299b2ca49658cd27900 P b84095e166154f4b93e54a628dca7381: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 16:52:05.610455 10728 log.cc:1196] T aa850cd81afe4299b2ca49658cd27900 P b84095e166154f4b93e54a628dca7381: Deleting WAL directory at /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786207919384338-8288-0/minicluster-data/ts-1/wal/wals/aa850cd81afe4299b2ca49658cd27900
I20260808 16:52:05.610692 10728 ts_tablet_manager.cc:1939] T aa850cd81afe4299b2ca49658cd27900 P b84095e166154f4b93e54a628dca7381: Deleting consensus metadata
I20260808 16:52:05.610967 9921 catalog_manager.cc:4841] TS b84095e166154f4b93e54a628dca7381 (127.8.24.2:41153): tablet aa850cd81afe4299b2ca49658cd27900 (table test-table [id=7eeb7cf9bf3f425f86a4ee5a30fa415f]) successfully deleted
Aug 08 16:52:05 dist-test-slave-6zbq krb5kdc[9872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207925, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.24.1@KRBTEST.COM
I20260808 16:52:05.624081 10698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:05.614595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35346 (local address 127.8.24.1:43441)
0808 16:52:05.614631 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:05.614633 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:05.614746 (+ 113us) server_negotiation.cc:407] Connection header received
0808 16:52:05.614794 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:05.614795 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:05.614803 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:05.614856 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:05.615507 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:05.616212 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:05.617290 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:05.617578 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:05.617689 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:05.620966 (+ 3277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:05.620974 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:05.620974 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:05.620981 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:05.622859 (+ 1878us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:05.623259 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:05.623262 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:05.623262 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:05.623343 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:05.623525 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:05.623526 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:05.623527 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:05.623691 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:05.623823 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:05.623893 (+ 70us) server_negotiation.cc:299] Negotiation successful
0808 16:52:05.623969 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
W20260808 16:52:05.624449 10230 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35346
I20260808 16:52:05.633526 10007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:05.626586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53198 (local address 127.8.24.62:38409)
0808 16:52:05.626636 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:05.626638 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:05.626703 (+ 65us) server_negotiation.cc:407] Connection header received
0808 16:52:05.626768 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:05.626769 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:05.626779 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:05.626832 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:05.627436 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:05.628193 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:05.629167 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:05.629467 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:05.629533 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:05.630848 (+ 1315us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:05.630849 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:05.630850 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:05.630857 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:05.632391 (+ 1534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:05.632763 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:05.632764 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:05.632764 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:05.632828 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:05.633014 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:05.633015 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:05.633015 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:05.633153 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:05.633257 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:05.633362 (+ 105us) server_negotiation.cc:299] Negotiation successful
0808 16:52:05.633439 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
W20260808 16:52:05.633709 9933 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53198
I20260808 16:52:05.634454 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 10047
I20260808 16:52:05.644261 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 10251
I20260808 16:52:05.654137 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 10473
I20260808 16:52:05.662444 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 9894
2026-08-08T16:52:05Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2212 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 16:52:05 dist-test-slave-6zbq krb5kdc[10795](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 16:52:05 dist-test-slave-6zbq krb5kdc[10795](info): set up 2 sockets
Aug 08 16:52:05 dist-test-slave-6zbq krb5kdc[10795](info): commencing operation
krb5kdc: starting...
W20260808 16:52:06.795610 8288 mini_kdc.cc:121] Time spent starting KDC: real 1.106s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/krb5kdc/test-user.keytab.
Aug 08 16:52:06 dist-test-slave-6zbq krb5kdc[10795](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207926, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T16:52:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T16:52:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.24.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
I20260808 16:52:06.900774 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:41799
--webserver_interface=127.8.24.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:38311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:41799
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:52:06.977982 10815 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:52:06.978157 10815 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:52:06.978178 10815 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:52:06.979236 10815 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 16:52:06.979269 10815 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 16:52:06.979281 10815 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:52:06.979292 10815 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:52:06.979310 10815 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 16:52:06.979321 10815 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 16:52:06.980660 10815 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:38311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:41799
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:41799
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.24.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10815
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:52:06.980835 10815 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:52:06.980993 10815 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:52:06.985284 10821 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:06.985308 10824 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:06.985311 10822 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:06.985407 10815 server_base.cc:1034] running on GCE node
I20260808 16:52:06.985692 10815 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:52:06.985929 10815 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:52:06.987071 10815 hybrid_clock.cc:648] HybridClock initialized: now 1786207926987055 us; error 36 us; skew 500 ppm
Aug 08 16:52:06 dist-test-slave-6zbq krb5kdc[10795](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207926, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:52:06.989311 10815 init.cc:377] Logged in from keytab as kudu/127.8.24.62@KRBTEST.COM (short username kudu)
I20260808 16:52:06.989697 10815 webserver.cc:460] Webserver started at http://127.8.24.62:41513/ using document root <none> and password file <none>
I20260808 16:52:06.989876 10815 fs_manager.cc:359] Metadata directory not provided
I20260808 16:52:06.989917 10815 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:52:06.990028 10815 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:52:06.990998 10815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/minicluster-data/master-0/data/instance:
uuid: "5627ffa85f6546a1a64643aa34ab1e6f"
format_stamp: "Formatted at 2026-08-08 16:52:06 on dist-test-slave-6zbq"
server_key: "5238f78e55a17a470c3344b3d189a899"
server_key_iv: "c5342a779ea54bebb2686f091494cb30"
server_key_version: "encryptionkey@0"
I20260808 16:52:06.991312 10815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/minicluster-data/master-0/wal/instance:
uuid: "5627ffa85f6546a1a64643aa34ab1e6f"
format_stamp: "Formatted at 2026-08-08 16:52:06 on dist-test-slave-6zbq"
server_key: "5238f78e55a17a470c3344b3d189a899"
server_key_iv: "c5342a779ea54bebb2686f091494cb30"
server_key_version: "encryptionkey@0"
I20260808 16:52:06.992456 10815 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 16:52:06.993047 10836 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:06.993170 10815 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 16:52:06.993258 10815 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/minicluster-data/master-0/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/minicluster-data/master-0/wal
uuid: "5627ffa85f6546a1a64643aa34ab1e6f"
format_stamp: "Formatted at 2026-08-08 16:52:06 on dist-test-slave-6zbq"
server_key: "5238f78e55a17a470c3344b3d189a899"
server_key_iv: "c5342a779ea54bebb2686f091494cb30"
server_key_version: "encryptionkey@0"
I20260808 16:52:06.993314 10815 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:52:07.005514 10815 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:52:07.007421 10815 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:52:07.007543 10815 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:52:07.011765 10815 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.62:41799
I20260808 16:52:07.011830 10929 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.62:41799 every 8 connection(s)
I20260808 16:52:07.012081 10815 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
I20260808 16:52:07.012658 10931 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:52:07.014698 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 10815
I20260808 16:52:07.014773 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/minicluster-data/master-0/wal/instance
I20260808 16:52:07.014804 10931 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5627ffa85f6546a1a64643aa34ab1e6f: Bootstrap starting.
I20260808 16:52:07.014918 8288 external_mini_cluster.cc:1428] Setting key 7812dda47f8b506d26196e99fba382b3
I20260808 16:52:07.015384 10931 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5627ffa85f6546a1a64643aa34ab1e6f: Neither blocks nor log segments found. Creating new log.
I20260808 16:52:07.015643 10931 log.cc:824] T 00000000000000000000000000000000 P 5627ffa85f6546a1a64643aa34ab1e6f: Log is configured to *not* fsync() on all Append() calls
I20260808 16:52:07.016263 10931 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5627ffa85f6546a1a64643aa34ab1e6f: No bootstrap required, opened a new log
I20260808 16:52:07.017536 10931 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5627ffa85f6546a1a64643aa34ab1e6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5627ffa85f6546a1a64643aa34ab1e6f" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 41799 } }
I20260808 16:52:07.017657 10931 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5627ffa85f6546a1a64643aa34ab1e6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:52:07.017688 10931 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5627ffa85f6546a1a64643aa34ab1e6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5627ffa85f6546a1a64643aa34ab1e6f, State: Initialized, Role: FOLLOWER
I20260808 16:52:07.017803 10931 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5627ffa85f6546a1a64643aa34ab1e6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5627ffa85f6546a1a64643aa34ab1e6f" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 41799 } }
I20260808 16:52:07.017874 10931 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5627ffa85f6546a1a64643aa34ab1e6f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 16:52:07.017905 10931 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5627ffa85f6546a1a64643aa34ab1e6f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 16:52:07.017951 10931 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5627ffa85f6546a1a64643aa34ab1e6f [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:52:07.018448 10931 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5627ffa85f6546a1a64643aa34ab1e6f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5627ffa85f6546a1a64643aa34ab1e6f" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 41799 } }
I20260808 16:52:07.018540 10931 leader_election.cc:302] T 00000000000000000000000000000000 P 5627ffa85f6546a1a64643aa34ab1e6f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5627ffa85f6546a1a64643aa34ab1e6f; no voters:
I20260808 16:52:07.018680 10931 leader_election.cc:288] T 00000000000000000000000000000000 P 5627ffa85f6546a1a64643aa34ab1e6f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 16:52:07.018735 10939 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5627ffa85f6546a1a64643aa34ab1e6f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:52:07.018900 10931 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5627ffa85f6546a1a64643aa34ab1e6f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 16:52:07.018903 10939 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5627ffa85f6546a1a64643aa34ab1e6f [term 1 LEADER]: Becoming Leader. State: Replica: 5627ffa85f6546a1a64643aa34ab1e6f, State: Running, Role: LEADER
I20260808 16:52:07.019086 10939 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5627ffa85f6546a1a64643aa34ab1e6f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5627ffa85f6546a1a64643aa34ab1e6f" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 41799 } }
I20260808 16:52:07.019382 10939 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5627ffa85f6546a1a64643aa34ab1e6f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5627ffa85f6546a1a64643aa34ab1e6f. Latest consensus state: current_term: 1 leader_uuid: "5627ffa85f6546a1a64643aa34ab1e6f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5627ffa85f6546a1a64643aa34ab1e6f" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 41799 } } }
I20260808 16:52:07.019452 10939 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5627ffa85f6546a1a64643aa34ab1e6f [sys.catalog]: This master's current role is: LEADER
I20260808 16:52:07.019594 10941 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5627ffa85f6546a1a64643aa34ab1e6f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5627ffa85f6546a1a64643aa34ab1e6f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5627ffa85f6546a1a64643aa34ab1e6f" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 41799 } } }
I20260808 16:52:07.019644 10941 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5627ffa85f6546a1a64643aa34ab1e6f [sys.catalog]: This master's current role is: LEADER
I20260808 16:52:07.019753 10950 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 16:52:07.020167 10950 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 16:52:07.021632 10950 catalog_manager.cc:1347] Generated new cluster ID: 94ae2b0d222843e29275ffa468c93b8d
I20260808 16:52:07.021687 10950 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 16:52:07 dist-test-slave-6zbq krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207926, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260808 16:52:07.025171 10936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:07.015336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59222 (local address 127.8.24.62:41799)
0808 16:52:07.015559 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:07.015567 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:07.015589 (+ 22us) server_negotiation.cc:407] Connection header received
0808 16:52:07.016112 (+ 523us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:07.016120 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:07.016166 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:07.016286 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:07.016866 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:07.018329 (+ 1463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:07.019292 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:07.019896 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:07.020014 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:07.022443 (+ 2429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:07.022447 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:07.022447 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:07.022459 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:07.023895 (+ 1436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:07.024202 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:07.024205 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:07.024205 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:07.024293 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:07.024464 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:07.024464 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:07.024465 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:07.024671 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:07.024833 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:07.024904 (+ 71us) server_negotiation.cc:299] Negotiation successful
0808 16:52:07.025027 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":75,"threads_started":1}
I20260808 16:52:07.039016 10950 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 16:52:07.039482 10950 catalog_manager.cc:1504] Loading token signing keys...
I20260808 16:52:07.044237 10950 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5627ffa85f6546a1a64643aa34ab1e6f: Generated new TSK 0
I20260808 16:52:07.044374 10950 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.24.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.1.keytab.
Entry for principal kudu/127.8.24.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.1.keytab.
I20260808 16:52:07.071450 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.1:0
--local_ip_for_outbound_sockets=127.8.24.1
--webserver_interface=127.8.24.1
--webserver_port=0
--tserver_master_addrs=127.8.24.62:41799
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.1.keytab
--principal=kudu/127.8.24.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:38311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:52:07.159571 10971 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:52:07.159755 10971 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:52:07.159780 10971 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:52:07.160866 10971 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:52:07.160905 10971 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:52:07.160926 10971 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 16:52:07.160949 10971 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 16:52:07.160979 10971 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.1
I20260808 16:52:07.162422 10971 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:38311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.1.keytab
--principal=kudu/127.8.24.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.24.1
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.24.62:41799
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10971
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.1
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:52:07.162623 10971 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:52:07.162840 10971 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:52:07.167322 10980 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:07.167335 10978 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:07.167495 10977 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:07.167555 10971 server_base.cc:1034] running on GCE node
I20260808 16:52:07.167905 10971 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:52:07.168169 10971 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:52:07.169334 10971 hybrid_clock.cc:648] HybridClock initialized: now 1786207927169321 us; error 29 us; skew 500 ppm
Aug 08 16:52:07 dist-test-slave-6zbq krb5kdc[10795](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207927, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:52:07.171738 10971 init.cc:377] Logged in from keytab as kudu/127.8.24.1@KRBTEST.COM (short username kudu)
I20260808 16:52:07.172132 10971 webserver.cc:460] Webserver started at http://127.8.24.1:35193/ using document root <none> and password file <none>
I20260808 16:52:07.172319 10971 fs_manager.cc:359] Metadata directory not provided
I20260808 16:52:07.172364 10971 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:52:07.172477 10971 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:52:07.173419 10971 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/minicluster-data/ts-0/data/instance:
uuid: "982252a2a0f04384a24931f2215efa1e"
format_stamp: "Formatted at 2026-08-08 16:52:07 on dist-test-slave-6zbq"
server_key: "b4c1a244a5ed8957d7bfbe482ea93844"
server_key_iv: "09b2d7a51472a677135297c68f367fbf"
server_key_version: "encryptionkey@0"
I20260808 16:52:07.173740 10971 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/minicluster-data/ts-0/wal/instance:
uuid: "982252a2a0f04384a24931f2215efa1e"
format_stamp: "Formatted at 2026-08-08 16:52:07 on dist-test-slave-6zbq"
server_key: "b4c1a244a5ed8957d7bfbe482ea93844"
server_key_iv: "09b2d7a51472a677135297c68f367fbf"
server_key_version: "encryptionkey@0"
I20260808 16:52:07.174966 10971 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 16:52:07.175554 10991 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:07.175737 10971 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 16:52:07.175825 10971 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/minicluster-data/ts-0/wal
uuid: "982252a2a0f04384a24931f2215efa1e"
format_stamp: "Formatted at 2026-08-08 16:52:07 on dist-test-slave-6zbq"
server_key: "b4c1a244a5ed8957d7bfbe482ea93844"
server_key_iv: "09b2d7a51472a677135297c68f367fbf"
server_key_version: "encryptionkey@0"
I20260808 16:52:07.175885 10971 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:52:07.189538 10971 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:52:07.191541 10971 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:52:07.191672 10971 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:52:07.192291 10971 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:52:07.192351 10971 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:07.192392 10971 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:52:07.192408 10971 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 16:52:07 dist-test-slave-6zbq krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207927, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.1@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260808 16:52:07.199137 10971 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.1:44087
I20260808 16:52:07.199512 10971 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/minicluster-data/ts-0/data/info.pb
I20260808 16:52:07.199926 11163 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.1:44087 every 8 connection(s)
I20260808 16:52:07.201458 10936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:07.192808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:60797 (local address 127.8.24.62:41799)
0808 16:52:07.192872 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:07.192874 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:07.193032 (+ 158us) server_negotiation.cc:407] Connection header received
0808 16:52:07.193925 (+ 893us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:07.193927 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:07.193940 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:07.194017 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:07.194712 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:07.195207 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:07.196423 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:07.196726 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:07.196783 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:07.199137 (+ 2354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:07.199138 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:07.199139 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:07.199147 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:07.200479 (+ 1332us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:07.200807 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:07.200808 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:07.200808 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:07.200853 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:07.201003 (+ 150us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:07.201004 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:07.201004 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:07.201105 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:07.201149 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:07.201334 (+ 185us) server_negotiation.cc:299] Negotiation successful
0808 16:52:07.201393 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 16:52:07.201608 11004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:07.192761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:41799 (local address 127.8.24.1:60797)
0808 16:52:07.192914 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:07.192952 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:07.193815 (+ 863us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:07.194090 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:07.194094 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:07.194441 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:07.194616 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:07.194622 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:07.195289 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:07.195290 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:07.196345 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:07.196348 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:07.196804 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:07.196805 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:07.196936 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:07.197431 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:07.197438 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:07.199046 (+ 1608us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:52:07.200545 (+ 1499us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:07.200548 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:07.200549 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:07.200753 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:07.200888 (+ 135us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:07.200889 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:07.200889 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:07.200965 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:07.201148 (+ 183us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:52:07.201151 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:52:07.201250 (+ 99us) client_negotiation.cc:769] Sending connection context
0808 16:52:07.201321 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 16:52:07.201439 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":86,"threads_started":1}
I20260808 16:52:07.202049 11166 heartbeater.cc:347] Connected to a master server at 127.8.24.62:41799
I20260808 16:52:07.202129 11166 heartbeater.cc:464] Registering TS with master...
I20260808 16:52:07.202294 11166 heartbeater.cc:511] Master 127.8.24.62:41799 requested a full tablet report, sending...
I20260808 16:52:07.202790 10857 ts_manager.cc:194] Registered new tserver with Master: 982252a2a0f04384a24931f2215efa1e (127.8.24.1:44087)
I20260808 16:52:07.203519 10857 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.1@KRBTEST.COM'} at 127.8.24.1:60797
I20260808 16:52:07.206166 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 10971
I20260808 16:52:07.206267 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/minicluster-data/ts-0/wal/instance
I20260808 16:52:07.206466 8288 external_mini_cluster.cc:1428] Setting key 9eeb886e8fc7a37dfd9594620483126e
I20260808 16:52:07.207857 10936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:07.203000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:33633 (local address 127.8.24.62:41799)
0808 16:52:07.203037 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:07.203038 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:07.203134 (+ 96us) server_negotiation.cc:407] Connection header received
0808 16:52:07.203176 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:07.203177 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:07.203186 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:07.203236 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:07.203487 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:07.204000 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:07.204713 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:07.204881 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:07.204918 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:07.205635 (+ 717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:07.205636 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:07.205636 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:07.205640 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:07.206860 (+ 1220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:07.207199 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:07.207200 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:07.207200 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:07.207253 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:07.207407 (+ 154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:07.207408 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:07.207408 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:07.207507 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:07.207557 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:07.207699 (+ 142us) server_negotiation.cc:299] Negotiation successful
0808 16:52:07.207769 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":8,"server-negotiator.queue_time_us":18}
I20260808 16:52:07.207871 11175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:07.202947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:41799 (local address 127.8.24.1:33633)
0808 16:52:07.203078 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:07.203087 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:07.203132 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:07.203242 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:07.203242 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:07.203385 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:07.203437 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:07.203438 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:07.204080 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:07.204081 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:07.204653 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:07.204655 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:07.204923 (+ 268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:07.204924 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:07.204972 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:07.205312 (+ 340us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:07.205316 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:07.205579 (+ 263us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:52:07.206977 (+ 1398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:07.206978 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:07.206979 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:07.207128 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:07.207304 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:07.207304 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:07.207305 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:07.207360 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:07.207558 (+ 198us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:52:07.207559 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:52:07.207600 (+ 41us) client_negotiation.cc:769] Sending connection context
0808 16:52:07.207669 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 16:52:07.207772 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":74,"threads_started":1}
WARNING: no policy specified for kudu/127.8.24.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.2.keytab.
Entry for principal kudu/127.8.24.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.2.keytab.
I20260808 16:52:07.228927 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.2:0
--local_ip_for_outbound_sockets=127.8.24.2
--webserver_interface=127.8.24.2
--webserver_port=0
--tserver_master_addrs=127.8.24.62:41799
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.2.keytab
--principal=kudu/127.8.24.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:38311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:52:07.306905 11179 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:52:07.307082 11179 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:52:07.307106 11179 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:52:07.308208 11179 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:52:07.308248 11179 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:52:07.308269 11179 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 16:52:07.308292 11179 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 16:52:07.308321 11179 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.2
I20260808 16:52:07.309733 11179 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:38311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.2.keytab
--principal=kudu/127.8.24.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.24.2
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.24.62:41799
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11179
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.2
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:52:07.309935 11179 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:52:07.310137 11179 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:52:07.314673 11187 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:07.314652 11188 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:07.314666 11191 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:07.314684 11179 server_base.cc:1034] running on GCE node
I20260808 16:52:07.315003 11179 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:52:07.315238 11179 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:52:07.316383 11179 hybrid_clock.cc:648] HybridClock initialized: now 1786207927316371 us; error 32 us; skew 500 ppm
Aug 08 16:52:07 dist-test-slave-6zbq krb5kdc[10795](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207927, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:52:07.318601 11179 init.cc:377] Logged in from keytab as kudu/127.8.24.2@KRBTEST.COM (short username kudu)
I20260808 16:52:07.319026 11179 webserver.cc:460] Webserver started at http://127.8.24.2:35143/ using document root <none> and password file <none>
I20260808 16:52:07.319242 11179 fs_manager.cc:359] Metadata directory not provided
I20260808 16:52:07.319283 11179 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:52:07.319392 11179 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:52:07.320350 11179 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/minicluster-data/ts-1/data/instance:
uuid: "74b028f8dcd8443493a06be381ff1317"
format_stamp: "Formatted at 2026-08-08 16:52:07 on dist-test-slave-6zbq"
server_key: "3c9cb5b62b82024640f99573c0049b78"
server_key_iv: "5002e7af64a9927fd7049bafb4000a61"
server_key_version: "encryptionkey@0"
I20260808 16:52:07.320654 11179 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/minicluster-data/ts-1/wal/instance:
uuid: "74b028f8dcd8443493a06be381ff1317"
format_stamp: "Formatted at 2026-08-08 16:52:07 on dist-test-slave-6zbq"
server_key: "3c9cb5b62b82024640f99573c0049b78"
server_key_iv: "5002e7af64a9927fd7049bafb4000a61"
server_key_version: "encryptionkey@0"
I20260808 16:52:07.321825 11179 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 16:52:07.322463 11199 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:07.322647 11179 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:07.322729 11179 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/minicluster-data/ts-1/wal
uuid: "74b028f8dcd8443493a06be381ff1317"
format_stamp: "Formatted at 2026-08-08 16:52:07 on dist-test-slave-6zbq"
server_key: "3c9cb5b62b82024640f99573c0049b78"
server_key_iv: "5002e7af64a9927fd7049bafb4000a61"
server_key_version: "encryptionkey@0"
I20260808 16:52:07.322796 11179 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:52:07.346113 11179 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:52:07.347736 11179 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:52:07.347855 11179 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:52:07.348325 11179 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:52:07.348363 11179 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:07.348382 11179 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:52:07.348393 11179 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 16:52:07 dist-test-slave-6zbq krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207927, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.2@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260808 16:52:07.354830 11179 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.2:44825
I20260808 16:52:07.354897 11388 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.2:44825 every 8 connection(s)
I20260808 16:52:07.355182 11179 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/minicluster-data/ts-1/data/info.pb
I20260808 16:52:07.357054 10936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:07.348707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:38903 (local address 127.8.24.62:41799)
0808 16:52:07.348774 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:07.348776 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:07.348921 (+ 145us) server_negotiation.cc:407] Connection header received
0808 16:52:07.349341 (+ 420us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:07.349342 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:07.349356 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:07.349419 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:07.350027 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:07.350652 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:07.351592 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:07.351784 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:07.351844 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:07.354664 (+ 2820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:07.354666 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:07.354666 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:07.354674 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:07.356033 (+ 1359us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:07.356381 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:07.356382 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:07.356382 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:07.356440 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:07.356614 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:07.356614 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:07.356614 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:07.356713 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:07.356757 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:07.356933 (+ 176us) server_negotiation.cc:299] Negotiation successful
0808 16:52:07.356992 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 16:52:07.357180 11215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:07.348655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:41799 (local address 127.8.24.2:38903)
0808 16:52:07.348804 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:07.348840 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:07.349257 (+ 417us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:07.349479 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:07.349482 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:07.349748 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:07.349931 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:07.349935 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:07.350736 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:07.350737 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:07.351503 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:07.351505 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:07.351858 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:07.351860 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:07.351992 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:07.352450 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:07.352461 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:07.354567 (+ 2106us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:52:07.356112 (+ 1545us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:07.356115 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:07.356116 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:07.356316 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:07.356490 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:07.356490 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:07.356491 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:07.356573 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:07.356748 (+ 175us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:52:07.356750 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:52:07.356847 (+ 97us) client_negotiation.cc:769] Sending connection context
0808 16:52:07.356916 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 16:52:07.357023 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":85,"threads_started":1}
I20260808 16:52:07.357602 11391 heartbeater.cc:347] Connected to a master server at 127.8.24.62:41799
I20260808 16:52:07.357676 11391 heartbeater.cc:464] Registering TS with master...
I20260808 16:52:07.357846 11391 heartbeater.cc:511] Master 127.8.24.62:41799 requested a full tablet report, sending...
I20260808 16:52:07.358217 10856 ts_manager.cc:194] Registered new tserver with Master: 74b028f8dcd8443493a06be381ff1317 (127.8.24.2:44825)
I20260808 16:52:07.358693 10856 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.2@KRBTEST.COM'} at 127.8.24.2:38903
I20260808 16:52:07.363040 10936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:07.358544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:57943 (local address 127.8.24.62:41799)
0808 16:52:07.358582 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:07.358584 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:07.358639 (+ 55us) server_negotiation.cc:407] Connection header received
0808 16:52:07.358701 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:07.358702 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:07.358709 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:07.358748 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:07.359078 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:07.359670 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:07.360276 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:07.360441 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:07.360480 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:07.361192 (+ 712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:07.361192 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:07.361192 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:07.361196 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:07.362151 (+ 955us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:07.362460 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:07.362461 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:07.362462 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:07.362518 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:07.362658 (+ 140us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:07.362659 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:07.362659 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:07.362785 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:07.362832 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:07.362911 (+ 79us) server_negotiation.cc:299] Negotiation successful
0808 16:52:07.362968 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 16:52:07.363035 11401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:07.358469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:41799 (local address 127.8.24.2:57943)
0808 16:52:07.358588 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:07.358598 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:07.358651 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:07.358763 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:07.358764 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:07.358945 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:07.359009 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:07.359010 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:07.359744 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:07.359745 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:07.360224 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:07.360225 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:07.360484 (+ 259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:07.360484 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:07.360531 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:07.360873 (+ 342us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:07.360878 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:07.361136 (+ 258us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:52:07.362214 (+ 1078us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:07.362215 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:07.362216 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:07.362387 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:07.362573 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:07.362573 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:07.362573 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:07.362618 (+ 45us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:07.362822 (+ 204us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:52:07.362823 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:52:07.362852 (+ 29us) client_negotiation.cc:769] Sending connection context
0808 16:52:07.362893 (+ 41us) client_negotiation.cc:240] Negotiation successful
0808 16:52:07.362957 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":69,"threads_started":1}
I20260808 16:52:07.363178 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 11179
I20260808 16:52:07.363270 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/minicluster-data/ts-1/wal/instance
I20260808 16:52:07.363423 8288 external_mini_cluster.cc:1428] Setting key 16b69f9c01a8286c6ad3bf59ea2eb152
WARNING: no policy specified for kudu/127.8.24.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.3.keytab.
Entry for principal kudu/127.8.24.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.3.keytab.
I20260808 16:52:07.386032 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.3:0
--local_ip_for_outbound_sockets=127.8.24.3
--webserver_interface=127.8.24.3
--webserver_port=0
--tserver_master_addrs=127.8.24.62:41799
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.3.keytab
--principal=kudu/127.8.24.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:38311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:52:07.464174 11410 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:52:07.464344 11410 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:52:07.464365 11410 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:52:07.465402 11410 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:52:07.465435 11410 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:52:07.465452 11410 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 16:52:07.465462 11410 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 16:52:07.465480 11410 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.3
I20260808 16:52:07.466845 11410 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:38311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.3.keytab
--principal=kudu/127.8.24.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.24.3
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.24.62:41799
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11410
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.3
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:52:07.467005 11410 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:52:07.467163 11410 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:52:07.471671 11415 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:07.471678 11417 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:07.471678 11419 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:07.471750 11410 server_base.cc:1034] running on GCE node
I20260808 16:52:07.472047 11410 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:52:07.472225 11410 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:52:07.473349 11410 hybrid_clock.cc:648] HybridClock initialized: now 1786207927473338 us; error 28 us; skew 500 ppm
Aug 08 16:52:07 dist-test-slave-6zbq krb5kdc[10795](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207927, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:52:07.475529 11410 init.cc:377] Logged in from keytab as kudu/127.8.24.3@KRBTEST.COM (short username kudu)
I20260808 16:52:07.475911 11410 webserver.cc:460] Webserver started at http://127.8.24.3:42815/ using document root <none> and password file <none>
I20260808 16:52:07.476073 11410 fs_manager.cc:359] Metadata directory not provided
I20260808 16:52:07.476109 11410 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:52:07.476197 11410 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:52:07.477087 11410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/minicluster-data/ts-2/data/instance:
uuid: "25f72b14a05d4ec6861e7a7bbbac9ef5"
format_stamp: "Formatted at 2026-08-08 16:52:07 on dist-test-slave-6zbq"
server_key: "5a92ff117c4d3172aa4fb152c2c9abfb"
server_key_iv: "8ef4ce816c5de0a3875f1f538005a04a"
server_key_version: "encryptionkey@0"
I20260808 16:52:07.477355 11410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/minicluster-data/ts-2/wal/instance:
uuid: "25f72b14a05d4ec6861e7a7bbbac9ef5"
format_stamp: "Formatted at 2026-08-08 16:52:07 on dist-test-slave-6zbq"
server_key: "5a92ff117c4d3172aa4fb152c2c9abfb"
server_key_iv: "8ef4ce816c5de0a3875f1f538005a04a"
server_key_version: "encryptionkey@0"
I20260808 16:52:07.478529 11410 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 16:52:07.479132 11431 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:07.479285 11410 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 16:52:07.479364 11410 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/minicluster-data/ts-2/wal
uuid: "25f72b14a05d4ec6861e7a7bbbac9ef5"
format_stamp: "Formatted at 2026-08-08 16:52:07 on dist-test-slave-6zbq"
server_key: "5a92ff117c4d3172aa4fb152c2c9abfb"
server_key_iv: "8ef4ce816c5de0a3875f1f538005a04a"
server_key_version: "encryptionkey@0"
I20260808 16:52:07.479415 11410 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:52:07.486390 11410 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:52:07.487854 11410 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:52:07.487947 11410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:52:07.488355 11410 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:52:07.488395 11410 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:07.488415 11410 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:52:07.488425 11410 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 16:52:07 dist-test-slave-6zbq krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207927, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.3@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260808 16:52:07.495150 11410 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.3:38397
I20260808 16:52:07.495190 11606 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.3:38397 every 8 connection(s)
I20260808 16:52:07.495493 11410 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/minicluster-data/ts-2/data/info.pb
I20260808 16:52:07.497049 10936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:07.488700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:49339 (local address 127.8.24.62:41799)
0808 16:52:07.488766 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:07.488768 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:07.488941 (+ 173us) server_negotiation.cc:407] Connection header received
0808 16:52:07.489508 (+ 567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:07.489509 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:07.489519 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:07.489553 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:07.490311 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:07.490854 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:07.491782 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:07.491970 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:07.492024 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:07.494529 (+ 2505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:07.494530 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:07.494531 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:07.494538 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:07.496080 (+ 1542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:07.496421 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:07.496422 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:07.496423 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:07.496478 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:07.496637 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:07.496637 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:07.496638 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:07.496730 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:07.496772 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:07.496937 (+ 165us) server_negotiation.cc:299] Negotiation successful
0808 16:52:07.496989 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 16:52:07.497191 11441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:07.488666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:41799 (local address 127.8.24.3:49339)
0808 16:52:07.488817 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:07.488861 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:07.489404 (+ 543us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:07.489640 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:07.489645 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:07.489958 (+ 313us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:07.490230 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:07.490236 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:07.490945 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:07.490946 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:07.491697 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:07.491699 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:07.492035 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:07.492036 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:07.492146 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:07.492562 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:07.492569 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:07.494451 (+ 1882us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:52:07.496144 (+ 1693us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:07.496148 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:07.496150 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:07.496358 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:07.496527 (+ 169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:07.496528 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:07.496528 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:07.496604 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:07.496765 (+ 161us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:52:07.496767 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:52:07.496860 (+ 93us) client_negotiation.cc:769] Sending connection context
0808 16:52:07.496925 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 16:52:07.497028 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":81,"threads_started":1}
I20260808 16:52:07.497566 11608 heartbeater.cc:347] Connected to a master server at 127.8.24.62:41799
I20260808 16:52:07.497642 11608 heartbeater.cc:464] Registering TS with master...
I20260808 16:52:07.497810 11608 heartbeater.cc:511] Master 127.8.24.62:41799 requested a full tablet report, sending...
I20260808 16:52:07.498169 10857 ts_manager.cc:194] Registered new tserver with Master: 25f72b14a05d4ec6861e7a7bbbac9ef5 (127.8.24.3:38397)
I20260808 16:52:07.498638 10857 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.3@KRBTEST.COM'} at 127.8.24.3:49339
I20260808 16:52:07.500003 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 11410
I20260808 16:52:07.500087 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/minicluster-data/ts-2/wal/instance
I20260808 16:52:07.500262 8288 external_mini_cluster.cc:1428] Setting key 70b8d53b56671b5880659b78e8e381d1
I20260808 16:52:07.501037 8288 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 16:52:07.503750 11619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:07.498516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:41799 (local address 127.8.24.3:49023)
0808 16:52:07.498653 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:07.498662 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:07.498723 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:07.498855 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:07.498856 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:07.499037 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:07.499095 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:07.499097 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:07.499687 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:07.499688 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:07.500222 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:07.500224 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:07.500608 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:07.500609 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:07.500686 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:07.501035 (+ 349us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:07.501040 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:07.501339 (+ 299us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:52:07.502846 (+ 1507us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:07.502846 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:07.502847 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:07.502991 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:07.503202 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:07.503203 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:07.503204 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:07.503268 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:07.503528 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:52:07.503528 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:52:07.503561 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 16:52:07.503607 (+ 46us) client_negotiation.cc:240] Negotiation successful
0808 16:52:07.503677 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":88,"threads_started":1}
I20260808 16:52:07.503782 10936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:07.498571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:49023 (local address 127.8.24.62:41799)
0808 16:52:07.498604 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:07.498605 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:07.498721 (+ 116us) server_negotiation.cc:407] Connection header received
0808 16:52:07.498775 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:07.498776 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:07.498784 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:07.498834 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:07.499175 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:07.499607 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:07.500292 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:07.500539 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:07.500598 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:07.501414 (+ 816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:07.501415 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:07.501416 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:07.501423 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:07.502777 (+ 1354us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:07.503051 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:07.503052 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:07.503052 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:07.503126 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:07.503338 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:07.503339 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:07.503339 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:07.503466 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:07.503532 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:07.503634 (+ 102us) server_negotiation.cc:299] Negotiation successful
0808 16:52:07.503707 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
Aug 08 16:52:07 dist-test-slave-6zbq krb5kdc[10795](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207927, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 16:52:07 dist-test-slave-6zbq krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207927, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260808 16:52:07.523167 10936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:07.517151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59230 (local address 127.8.24.62:41799)
0808 16:52:07.517197 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:07.517198 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:07.517205 (+ 7us) server_negotiation.cc:407] Connection header received
0808 16:52:07.517256 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:07.517257 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:07.517263 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:07.517308 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:07.517774 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:07.518210 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:07.518902 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:07.519071 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:07.519110 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:07.521106 (+ 1996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:07.521108 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:07.521108 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:07.521113 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:07.522225 (+ 1112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:07.522542 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:07.522543 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:07.522543 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:07.522588 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:07.522754 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:07.522755 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:07.522756 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:07.522845 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:07.522922 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:07.523024 (+ 102us) server_negotiation.cc:299] Negotiation successful
0808 16:52:07.523088 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 16:52:07.524390 10857 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59230:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 16:52:07.526006 10857 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 16:52:07.531852 11632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:07.527844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:44087 (local address 127.0.0.1:43148)
0808 16:52:07.527984 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:07.528027 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:07.528072 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:07.528566 (+ 494us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:07.528566 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:07.528568 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:52:07.528604 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:07.528605 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:07.529814 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:07.529815 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:07.530660 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:07.530663 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:07.531555 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:07.531556 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:07.531636 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:07.531648 (+ 12us) client_negotiation.cc:769] Sending connection context
0808 16:52:07.531715 (+ 67us) client_negotiation.cc:240] Negotiation successful
0808 16:52:07.531779 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"spinlock_wait_cycles":768,"thread_start_us":47,"threads_started":1}
I20260808 16:52:07.531880 11638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:07.528032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43148 (local address 127.8.24.1:44087)
0808 16:52:07.528299 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:07.528302 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:07.528312 (+ 10us) server_negotiation.cc:407] Connection header received
0808 16:52:07.528341 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:07.528342 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:07.528353 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:07.528417 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:52:07.528689 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:07.529744 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:07.530768 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:07.531399 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:07.531444 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:07.531494 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:07.531746 (+ 252us) server_negotiation.cc:299] Negotiation successful
0808 16:52:07.531787 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":240,"thread_start_us":65,"threads_started":1}
I20260808 16:52:07.532045 11634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:07.527841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.2:44825 (local address 127.0.0.1:51682)
0808 16:52:07.527981 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:07.528013 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:07.528072 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:07.528656 (+ 584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:07.528657 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:07.528659 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:52:07.528704 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:07.528706 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:07.529589 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:07.529590 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:07.530927 (+ 1337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:07.530929 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:07.531814 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:07.531815 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:07.531878 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:07.531882 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 16:52:07.531936 (+ 54us) client_negotiation.cc:240] Negotiation successful
0808 16:52:07.531976 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"mutex_wait_us":36,"thread_start_us":51,"threads_started":1}
I20260808 16:52:07.532307 11635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:07.528002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51682 (local address 127.8.24.2:44825)
0808 16:52:07.528502 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:07.528506 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:07.528512 (+ 6us) server_negotiation.cc:407] Connection header received
0808 16:52:07.528538 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:07.528539 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:07.528549 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:07.528778 (+ 229us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:52:07.528837 (+ 59us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:07.529511 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:07.531009 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:07.531735 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:07.531796 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:07.531848 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:07.532200 (+ 352us) server_negotiation.cc:299] Negotiation successful
0808 16:52:07.532239 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":473,"thread_start_us":62,"threads_started":1}
I20260808 16:52:07.532526 11060 tablet_service.cc:1467] Processing CreateTablet for tablet fbcb3fc43b2d4448b28a5b7b7e4b4f65 (DEFAULT_TABLE table=test-table [id=4c0d4216b57b44dd9ce83624024571db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:52:07.532900 11060 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fbcb3fc43b2d4448b28a5b7b7e4b4f65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:52:07.532820 11278 tablet_service.cc:1467] Processing CreateTablet for tablet fbcb3fc43b2d4448b28a5b7b7e4b4f65 (DEFAULT_TABLE table=test-table [id=4c0d4216b57b44dd9ce83624024571db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:52:07.533133 11278 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fbcb3fc43b2d4448b28a5b7b7e4b4f65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:52:07.533309 11633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:07.527853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.3:38397 (local address 127.0.0.1:50978)
0808 16:52:07.528173 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:07.528181 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:07.528214 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:07.528434 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:07.528436 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:07.528438 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:52:07.528493 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:07.528494 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:07.529659 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:07.529660 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:07.530925 (+ 1265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:07.530929 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:07.533028 (+ 2099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:07.533030 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:07.533090 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:07.533095 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 16:52:07.533150 (+ 55us) client_negotiation.cc:240] Negotiation successful
0808 16:52:07.533245 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"mutex_wait_us":44,"thread_start_us":47,"threads_started":1}
I20260808 16:52:07.533536 11641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:07.528162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50978 (local address 127.8.24.3:38397)
0808 16:52:07.528307 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:07.528311 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:07.528318 (+ 7us) server_negotiation.cc:407] Connection header received
0808 16:52:07.528344 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:07.528345 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:07.528356 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:07.528418 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:52:07.528554 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:07.529584 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:07.531882 (+ 2298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:07.532861 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:07.532925 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:07.532978 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:07.533410 (+ 432us) server_negotiation.cc:299] Negotiation successful
0808 16:52:07.533445 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":65,"threads_started":1}
I20260808 16:52:07.535058 11501 tablet_service.cc:1467] Processing CreateTablet for tablet fbcb3fc43b2d4448b28a5b7b7e4b4f65 (DEFAULT_TABLE table=test-table [id=4c0d4216b57b44dd9ce83624024571db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:52:07.535380 11501 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fbcb3fc43b2d4448b28a5b7b7e4b4f65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:52:07.536211 11643 tablet_bootstrap.cc:492] T fbcb3fc43b2d4448b28a5b7b7e4b4f65 P 982252a2a0f04384a24931f2215efa1e: Bootstrap starting.
I20260808 16:52:07.536715 11643 tablet_bootstrap.cc:654] T fbcb3fc43b2d4448b28a5b7b7e4b4f65 P 982252a2a0f04384a24931f2215efa1e: Neither blocks nor log segments found. Creating new log.
I20260808 16:52:07.536928 11643 log.cc:824] T fbcb3fc43b2d4448b28a5b7b7e4b4f65 P 982252a2a0f04384a24931f2215efa1e: Log is configured to *not* fsync() on all Append() calls
I20260808 16:52:07.537532 11643 tablet_bootstrap.cc:492] T fbcb3fc43b2d4448b28a5b7b7e4b4f65 P 982252a2a0f04384a24931f2215efa1e: No bootstrap required, opened a new log
I20260808 16:52:07.537575 11643 ts_tablet_manager.cc:1397] T fbcb3fc43b2d4448b28a5b7b7e4b4f65 P 982252a2a0f04384a24931f2215efa1e: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 16:52:07.537905 11645 tablet_bootstrap.cc:492] T fbcb3fc43b2d4448b28a5b7b7e4b4f65 P 25f72b14a05d4ec6861e7a7bbbac9ef5: Bootstrap starting.
I20260808 16:52:07.538146 11642 tablet_bootstrap.cc:492] T fbcb3fc43b2d4448b28a5b7b7e4b4f65 P 74b028f8dcd8443493a06be381ff1317: Bootstrap starting.
I20260808 16:52:07.538645 11645 tablet_bootstrap.cc:654] T fbcb3fc43b2d4448b28a5b7b7e4b4f65 P 25f72b14a05d4ec6861e7a7bbbac9ef5: Neither blocks nor log segments found. Creating new log.
I20260808 16:52:07.538769 11642 tablet_bootstrap.cc:654] T fbcb3fc43b2d4448b28a5b7b7e4b4f65 P 74b028f8dcd8443493a06be381ff1317: Neither blocks nor log segments found. Creating new log.
I20260808 16:52:07.538954 11645 log.cc:824] T fbcb3fc43b2d4448b28a5b7b7e4b4f65 P 25f72b14a05d4ec6861e7a7bbbac9ef5: Log is configured to *not* fsync() on all Append() calls
I20260808 16:52:07.539074 11642 log.cc:824] T fbcb3fc43b2d4448b28a5b7b7e4b4f65 P 74b028f8dcd8443493a06be381ff1317: Log is configured to *not* fsync() on all Append() calls
I20260808 16:52:07.539367 11643 raft_consensus.cc:348] T fbcb3fc43b2d4448b28a5b7b7e4b4f65 P 982252a2a0f04384a24931f2215efa1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "982252a2a0f04384a24931f2215efa1e" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44087 } } peers { permanent_uuid: "74b028f8dcd8443493a06be381ff1317" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 44825 } } peers { permanent_uuid: "25f72b14a05d4ec6861e7a7bbbac9ef5" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 38397 } }
I20260808 16:52:07.539487 11643 raft_consensus.cc:374] T fbcb3fc43b2d4448b28a5b7b7e4b4f65 P 982252a2a0f04384a24931f2215efa1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:52:07.539520 11643 raft_consensus.cc:729] T fbcb3fc43b2d4448b28a5b7b7e4b4f65 P 982252a2a0f04384a24931f2215efa1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 982252a2a0f04384a24931f2215efa1e, State: Initialized, Role: FOLLOWER
I20260808 16:52:07.539626 11645 tablet_bootstrap.cc:492] T fbcb3fc43b2d4448b28a5b7b7e4b4f65 P 25f72b14a05d4ec6861e7a7bbbac9ef5: No bootstrap required, opened a new log
I20260808 16:52:07.539628 11643 consensus_queue.cc:260] T fbcb3fc43b2d4448b28a5b7b7e4b4f65 P 982252a2a0f04384a24931f2215efa1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "982252a2a0f04384a24931f2215efa1e" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44087 } } peers { permanent_uuid: "74b028f8dcd8443493a06be381ff1317" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 44825 } } peers { permanent_uuid: "25f72b14a05d4ec6861e7a7bbbac9ef5" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 38397 } }
I20260808 16:52:07.539682 11645 ts_tablet_manager.cc:1397] T fbcb3fc43b2d4448b28a5b7b7e4b4f65 P 25f72b14a05d4ec6861e7a7bbbac9ef5: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 16:52:07.539809 11642 tablet_bootstrap.cc:492] T fbcb3fc43b2d4448b28a5b7b7e4b4f65 P 74b028f8dcd8443493a06be381ff1317: No bootstrap required, opened a new log
I20260808 16:52:07.539862 11642 ts_tablet_manager.cc:1397] T fbcb3fc43b2d4448b28a5b7b7e4b4f65 P 74b028f8dcd8443493a06be381ff1317: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 16:52:07.539878 11643 ts_tablet_manager.cc:1428] T fbcb3fc43b2d4448b28a5b7b7e4b4f65 P 982252a2a0f04384a24931f2215efa1e: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 16:52:07.539912 11166 heartbeater.cc:503] Master 127.8.24.62:41799 was elected leader, sending a full tablet report...
I20260808 16:52:07.541316 11645 raft_consensus.cc:348] T fbcb3fc43b2d4448b28a5b7b7e4b4f65 P 25f72b14a05d4ec6861e7a7bbbac9ef5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "982252a2a0f04384a24931f2215efa1e" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44087 } } peers { permanent_uuid: "74b028f8dcd8443493a06be381ff1317" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 44825 } } peers { permanent_uuid: "25f72b14a05d4ec6861e7a7bbbac9ef5" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 38397 } }
I20260808 16:52:07.541420 11645 raft_consensus.cc:374] T fbcb3fc43b2d4448b28a5b7b7e4b4f65 P 25f72b14a05d4ec6861e7a7bbbac9ef5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:52:07.541437 11645 raft_consensus.cc:729] T fbcb3fc43b2d4448b28a5b7b7e4b4f65 P 25f72b14a05d4ec6861e7a7bbbac9ef5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25f72b14a05d4ec6861e7a7bbbac9ef5, State: Initialized, Role: FOLLOWER
I20260808 16:52:07.541447 11642 raft_consensus.cc:348] T fbcb3fc43b2d4448b28a5b7b7e4b4f65 P 74b028f8dcd8443493a06be381ff1317 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "982252a2a0f04384a24931f2215efa1e" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44087 } } peers { permanent_uuid: "74b028f8dcd8443493a06be381ff1317" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 44825 } } peers { permanent_uuid: "25f72b14a05d4ec6861e7a7bbbac9ef5" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 38397 } }
I20260808 16:52:07.541533 11645 consensus_queue.cc:260] T fbcb3fc43b2d4448b28a5b7b7e4b4f65 P 25f72b14a05d4ec6861e7a7bbbac9ef5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "982252a2a0f04384a24931f2215efa1e" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44087 } } peers { permanent_uuid: "74b028f8dcd8443493a06be381ff1317" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 44825 } } peers { permanent_uuid: "25f72b14a05d4ec6861e7a7bbbac9ef5" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 38397 } }
I20260808 16:52:07.541576 11642 raft_consensus.cc:374] T fbcb3fc43b2d4448b28a5b7b7e4b4f65 P 74b028f8dcd8443493a06be381ff1317 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:52:07.541611 11642 raft_consensus.cc:729] T fbcb3fc43b2d4448b28a5b7b7e4b4f65 P 74b028f8dcd8443493a06be381ff1317 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74b028f8dcd8443493a06be381ff1317, State: Initialized, Role: FOLLOWER
I20260808 16:52:07.541718 11642 consensus_queue.cc:260] T fbcb3fc43b2d4448b28a5b7b7e4b4f65 P 74b028f8dcd8443493a06be381ff1317 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "982252a2a0f04384a24931f2215efa1e" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44087 } } peers { permanent_uuid: "74b028f8dcd8443493a06be381ff1317" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 44825 } } peers { permanent_uuid: "25f72b14a05d4ec6861e7a7bbbac9ef5" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 38397 } }
I20260808 16:52:07.541777 11645 ts_tablet_manager.cc:1428] T fbcb3fc43b2d4448b28a5b7b7e4b4f65 P 25f72b14a05d4ec6861e7a7bbbac9ef5: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260808 16:52:07.541906 11608 heartbeater.cc:503] Master 127.8.24.62:41799 was elected leader, sending a full tablet report...
I20260808 16:52:07.541954 11642 ts_tablet_manager.cc:1428] T fbcb3fc43b2d4448b28a5b7b7e4b4f65 P 74b028f8dcd8443493a06be381ff1317: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 16:52:07.542068 11391 heartbeater.cc:503] Master 127.8.24.62:41799 was elected leader, sending a full tablet report...
W20260808 16:52:07.605669 11392 tablet.cc:2369] T fbcb3fc43b2d4448b28a5b7b7e4b4f65 P 74b028f8dcd8443493a06be381ff1317: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 16:52:07.679031 11651 raft_consensus.cc:482] T fbcb3fc43b2d4448b28a5b7b7e4b4f65 P 25f72b14a05d4ec6861e7a7bbbac9ef5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 16:52:07.679133 11651 raft_consensus.cc:504] T fbcb3fc43b2d4448b28a5b7b7e4b4f65 P 25f72b14a05d4ec6861e7a7bbbac9ef5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "982252a2a0f04384a24931f2215efa1e" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44087 } } peers { permanent_uuid: "74b028f8dcd8443493a06be381ff1317" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 44825 } } peers { permanent_uuid: "25f72b14a05d4ec6861e7a7bbbac9ef5" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 38397 } }
I20260808 16:52:07.679461 11651 leader_election.cc:288] T fbcb3fc43b2d4448b28a5b7b7e4b4f65 P 25f72b14a05d4ec6861e7a7bbbac9ef5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 982252a2a0f04384a24931f2215efa1e (127.8.24.1:44087), 74b028f8dcd8443493a06be381ff1317 (127.8.24.2:44825)
I20260808 16:52:07.682160 11441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:07.679623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:44087 (local address 127.8.24.3:45447)
0808 16:52:07.679673 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:07.679685 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:07.679730 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:07.679872 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:07.679872 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:07.679874 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:52:07.679944 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:07.679945 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:07.680575 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:07.680576 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:07.681372 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:07.681374 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:07.681933 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:07.681934 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:07.682007 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:07.682010 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 16:52:07.682054 (+ 44us) client_negotiation.cc:240] Negotiation successful
0808 16:52:07.682091 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30,"mutex_wait_us":25}
I20260808 16:52:07.682171 11638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:07.679719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:45447 (local address 127.8.24.1:44087)
0808 16:52:07.679754 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:07.679756 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:07.679763 (+ 7us) server_negotiation.cc:407] Connection header received
0808 16:52:07.679789 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:07.679790 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:07.679800 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:07.679857 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:52:07.680007 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:07.680507 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:07.681453 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:07.681859 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:07.681918 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:07.681929 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:07.682067 (+ 138us) server_negotiation.cc:299] Negotiation successful
0808 16:52:07.682107 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 16:52:07.682226 11635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:07.679700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:56309 (local address 127.8.24.2:44825)
0808 16:52:07.679743 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:07.679744 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:07.679901 (+ 157us) server_negotiation.cc:407] Connection header received
0808 16:52:07.679939 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:07.679940 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:07.679951 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:07.680003 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:52:07.680144 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:07.680601 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:07.681473 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:07.681885 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:07.681943 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:07.681952 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:07.682124 (+ 172us) server_negotiation.cc:299] Negotiation successful
0808 16:52:07.682167 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 16:52:07.682453 11655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:07.679626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.2:44825 (local address 127.8.24.3:56309)
0808 16:52:07.679833 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:07.679843 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:07.679898 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:07.680019 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:07.680020 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:07.680021 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:52:07.680086 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:07.680088 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:07.680672 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:07.680672 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:07.681392 (+ 720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:07.681394 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:07.681963 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:07.681964 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:07.682030 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:07.682034 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 16:52:07.682111 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 16:52:07.682145 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"mutex_wait_us":53,"thread_start_us":90,"threads_started":1}
I20260808 16:52:07.682495 11092 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbcb3fc43b2d4448b28a5b7b7e4b4f65" candidate_uuid: "25f72b14a05d4ec6861e7a7bbbac9ef5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "982252a2a0f04384a24931f2215efa1e" is_pre_election: true
I20260808 16:52:07.682642 11092 raft_consensus.cc:2454] T fbcb3fc43b2d4448b28a5b7b7e4b4f65 P 982252a2a0f04384a24931f2215efa1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 25f72b14a05d4ec6861e7a7bbbac9ef5 in term 0.
I20260808 16:52:07.682718 11309 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbcb3fc43b2d4448b28a5b7b7e4b4f65" candidate_uuid: "25f72b14a05d4ec6861e7a7bbbac9ef5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74b028f8dcd8443493a06be381ff1317" is_pre_election: true
I20260808 16:52:07.682849 11309 raft_consensus.cc:2454] T fbcb3fc43b2d4448b28a5b7b7e4b4f65 P 74b028f8dcd8443493a06be381ff1317 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 25f72b14a05d4ec6861e7a7bbbac9ef5 in term 0.
I20260808 16:52:07.682854 11434 leader_election.cc:302] T fbcb3fc43b2d4448b28a5b7b7e4b4f65 P 25f72b14a05d4ec6861e7a7bbbac9ef5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25f72b14a05d4ec6861e7a7bbbac9ef5, 982252a2a0f04384a24931f2215efa1e; no voters:
I20260808 16:52:07.682955 11651 raft_consensus.cc:2781] T fbcb3fc43b2d4448b28a5b7b7e4b4f65 P 25f72b14a05d4ec6861e7a7bbbac9ef5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 16:52:07.683008 11651 raft_consensus.cc:482] T fbcb3fc43b2d4448b28a5b7b7e4b4f65 P 25f72b14a05d4ec6861e7a7bbbac9ef5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 16:52:07.683028 11651 raft_consensus.cc:3037] T fbcb3fc43b2d4448b28a5b7b7e4b4f65 P 25f72b14a05d4ec6861e7a7bbbac9ef5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:52:07.683686 11651 raft_consensus.cc:504] T fbcb3fc43b2d4448b28a5b7b7e4b4f65 P 25f72b14a05d4ec6861e7a7bbbac9ef5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "982252a2a0f04384a24931f2215efa1e" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44087 } } peers { permanent_uuid: "74b028f8dcd8443493a06be381ff1317" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 44825 } } peers { permanent_uuid: "25f72b14a05d4ec6861e7a7bbbac9ef5" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 38397 } }
I20260808 16:52:07.683799 11651 leader_election.cc:288] T fbcb3fc43b2d4448b28a5b7b7e4b4f65 P 25f72b14a05d4ec6861e7a7bbbac9ef5 [CANDIDATE]: Term 1 election: Requested vote from peers 982252a2a0f04384a24931f2215efa1e (127.8.24.1:44087), 74b028f8dcd8443493a06be381ff1317 (127.8.24.2:44825)
I20260808 16:52:07.683933 11092 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbcb3fc43b2d4448b28a5b7b7e4b4f65" candidate_uuid: "25f72b14a05d4ec6861e7a7bbbac9ef5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "982252a2a0f04384a24931f2215efa1e"
I20260808 16:52:07.683943 11309 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbcb3fc43b2d4448b28a5b7b7e4b4f65" candidate_uuid: "25f72b14a05d4ec6861e7a7bbbac9ef5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74b028f8dcd8443493a06be381ff1317"
I20260808 16:52:07.684007 11092 raft_consensus.cc:3037] T fbcb3fc43b2d4448b28a5b7b7e4b4f65 P 982252a2a0f04384a24931f2215efa1e [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:52:07.684007 11309 raft_consensus.cc:3037] T fbcb3fc43b2d4448b28a5b7b7e4b4f65 P 74b028f8dcd8443493a06be381ff1317 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:52:07.684623 11309 raft_consensus.cc:2454] T fbcb3fc43b2d4448b28a5b7b7e4b4f65 P 74b028f8dcd8443493a06be381ff1317 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25f72b14a05d4ec6861e7a7bbbac9ef5 in term 1.
I20260808 16:52:07.684633 11092 raft_consensus.cc:2454] T fbcb3fc43b2d4448b28a5b7b7e4b4f65 P 982252a2a0f04384a24931f2215efa1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25f72b14a05d4ec6861e7a7bbbac9ef5 in term 1.
I20260808 16:52:07.685050 11432 leader_election.cc:302] T fbcb3fc43b2d4448b28a5b7b7e4b4f65 P 25f72b14a05d4ec6861e7a7bbbac9ef5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25f72b14a05d4ec6861e7a7bbbac9ef5, 74b028f8dcd8443493a06be381ff1317; no voters:
I20260808 16:52:07.685150 11651 raft_consensus.cc:2781] T fbcb3fc43b2d4448b28a5b7b7e4b4f65 P 25f72b14a05d4ec6861e7a7bbbac9ef5 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:52:07.685274 11651 raft_consensus.cc:686] T fbcb3fc43b2d4448b28a5b7b7e4b4f65 P 25f72b14a05d4ec6861e7a7bbbac9ef5 [term 1 LEADER]: Becoming Leader. State: Replica: 25f72b14a05d4ec6861e7a7bbbac9ef5, State: Running, Role: LEADER
I20260808 16:52:07.685354 11651 consensus_queue.cc:237] T fbcb3fc43b2d4448b28a5b7b7e4b4f65 P 25f72b14a05d4ec6861e7a7bbbac9ef5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "982252a2a0f04384a24931f2215efa1e" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44087 } } peers { permanent_uuid: "74b028f8dcd8443493a06be381ff1317" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 44825 } } peers { permanent_uuid: "25f72b14a05d4ec6861e7a7bbbac9ef5" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 38397 } }
I20260808 16:52:07.686082 10857 catalog_manager.cc:5495] T fbcb3fc43b2d4448b28a5b7b7e4b4f65 P 25f72b14a05d4ec6861e7a7bbbac9ef5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 25f72b14a05d4ec6861e7a7bbbac9ef5 (127.8.24.3). New cstate: current_term: 1 leader_uuid: "25f72b14a05d4ec6861e7a7bbbac9ef5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "982252a2a0f04384a24931f2215efa1e" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74b028f8dcd8443493a06be381ff1317" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 44825 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25f72b14a05d4ec6861e7a7bbbac9ef5" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 38397 } health_report { overall_health: HEALTHY } } }
W20260808 16:52:07.700366 11167 tablet.cc:2369] T fbcb3fc43b2d4448b28a5b7b7e4b4f65 P 982252a2a0f04384a24931f2215efa1e: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 08 16:52:07 dist-test-slave-6zbq krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207926, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260808 16:52:07.732942 10936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:07.726986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59236 (local address 127.8.24.62:41799)
0808 16:52:07.727032 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:07.727034 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:07.727182 (+ 148us) server_negotiation.cc:407] Connection header received
0808 16:52:07.727243 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:07.727244 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:07.727254 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:07.727300 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:07.727731 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:07.728204 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:07.728846 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:07.729014 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:07.729066 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:07.730925 (+ 1859us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:07.730926 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:07.730927 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:07.730934 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:07.731952 (+ 1018us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:07.732278 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:07.732279 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:07.732280 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:07.732327 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:07.732481 (+ 154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:07.732481 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:07.732482 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:07.732579 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:07.732638 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:07.732775 (+ 137us) server_negotiation.cc:299] Negotiation successful
0808 16:52:07.732849 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 16:52:07.733017 11669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:07.726939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:41799 (local address 127.0.0.1:59236)
0808 16:52:07.727117 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:07.727127 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:07.727190 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:07.727312 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:07.727313 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:07.727613 (+ 300us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:07.727657 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:07.727660 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:07.728270 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:07.728271 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:07.728780 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:07.728781 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:07.729070 (+ 289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:07.729070 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:07.729121 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:07.729556 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:07.729560 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:07.730850 (+ 1290us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:52:07.732022 (+ 1172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:07.732029 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:07.732032 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:07.732215 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:07.732378 (+ 163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:07.732379 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:07.732379 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:07.732432 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:07.732633 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:52:07.732640 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:52:07.732687 (+ 47us) client_negotiation.cc:769] Sending connection context
0808 16:52:07.732745 (+ 58us) client_negotiation.cc:240] Negotiation successful
0808 16:52:07.732914 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"mutex_wait_us":72,"thread_start_us":94,"threads_started":1}
I20260808 16:52:07.734831 10857 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:59236:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 16:52:07.735013 10857 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 16:52:07.736353 11278 tablet_service.cc:1467] Processing CreateTablet for tablet 3b3a66cee0b545dd9694e383f2069f6f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e39828e39e33472782a35dd8913b98d2]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 16:52:07.736390 11501 tablet_service.cc:1467] Processing CreateTablet for tablet 3b3a66cee0b545dd9694e383f2069f6f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e39828e39e33472782a35dd8913b98d2]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 16:52:07.736523 11278 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b3a66cee0b545dd9694e383f2069f6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:52:07.736523 11501 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b3a66cee0b545dd9694e383f2069f6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:52:07.737978 11645 tablet_bootstrap.cc:492] T 3b3a66cee0b545dd9694e383f2069f6f P 25f72b14a05d4ec6861e7a7bbbac9ef5: Bootstrap starting.
I20260808 16:52:07.737978 11642 tablet_bootstrap.cc:492] T 3b3a66cee0b545dd9694e383f2069f6f P 74b028f8dcd8443493a06be381ff1317: Bootstrap starting.
I20260808 16:52:07.738056 11060 tablet_service.cc:1467] Processing CreateTablet for tablet 3b3a66cee0b545dd9694e383f2069f6f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e39828e39e33472782a35dd8913b98d2]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 16:52:07.738197 11060 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b3a66cee0b545dd9694e383f2069f6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:52:07.738458 11645 tablet_bootstrap.cc:654] T 3b3a66cee0b545dd9694e383f2069f6f P 25f72b14a05d4ec6861e7a7bbbac9ef5: Neither blocks nor log segments found. Creating new log.
I20260808 16:52:07.738749 11642 tablet_bootstrap.cc:654] T 3b3a66cee0b545dd9694e383f2069f6f P 74b028f8dcd8443493a06be381ff1317: Neither blocks nor log segments found. Creating new log.
I20260808 16:52:07.739133 11645 tablet_bootstrap.cc:492] T 3b3a66cee0b545dd9694e383f2069f6f P 25f72b14a05d4ec6861e7a7bbbac9ef5: No bootstrap required, opened a new log
I20260808 16:52:07.739177 11645 ts_tablet_manager.cc:1397] T 3b3a66cee0b545dd9694e383f2069f6f P 25f72b14a05d4ec6861e7a7bbbac9ef5: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260808 16:52:07.739337 11645 raft_consensus.cc:348] T 3b3a66cee0b545dd9694e383f2069f6f P 25f72b14a05d4ec6861e7a7bbbac9ef5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "982252a2a0f04384a24931f2215efa1e" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44087 } } peers { permanent_uuid: "74b028f8dcd8443493a06be381ff1317" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 44825 } } peers { permanent_uuid: "25f72b14a05d4ec6861e7a7bbbac9ef5" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 38397 } }
I20260808 16:52:07.739400 11642 tablet_bootstrap.cc:492] T 3b3a66cee0b545dd9694e383f2069f6f P 74b028f8dcd8443493a06be381ff1317: No bootstrap required, opened a new log
I20260808 16:52:07.739409 11645 raft_consensus.cc:374] T 3b3a66cee0b545dd9694e383f2069f6f P 25f72b14a05d4ec6861e7a7bbbac9ef5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:52:07.739430 11645 raft_consensus.cc:729] T 3b3a66cee0b545dd9694e383f2069f6f P 25f72b14a05d4ec6861e7a7bbbac9ef5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25f72b14a05d4ec6861e7a7bbbac9ef5, State: Initialized, Role: FOLLOWER
I20260808 16:52:07.739440 11642 ts_tablet_manager.cc:1397] T 3b3a66cee0b545dd9694e383f2069f6f P 74b028f8dcd8443493a06be381ff1317: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260808 16:52:07.739475 11643 tablet_bootstrap.cc:492] T 3b3a66cee0b545dd9694e383f2069f6f P 982252a2a0f04384a24931f2215efa1e: Bootstrap starting.
I20260808 16:52:07.739470 11645 consensus_queue.cc:260] T 3b3a66cee0b545dd9694e383f2069f6f P 25f72b14a05d4ec6861e7a7bbbac9ef5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "982252a2a0f04384a24931f2215efa1e" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44087 } } peers { permanent_uuid: "74b028f8dcd8443493a06be381ff1317" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 44825 } } peers { permanent_uuid: "25f72b14a05d4ec6861e7a7bbbac9ef5" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 38397 } }
I20260808 16:52:07.739563 11645 ts_tablet_manager.cc:1428] T 3b3a66cee0b545dd9694e383f2069f6f P 25f72b14a05d4ec6861e7a7bbbac9ef5: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:07.739578 11642 raft_consensus.cc:348] T 3b3a66cee0b545dd9694e383f2069f6f P 74b028f8dcd8443493a06be381ff1317 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "982252a2a0f04384a24931f2215efa1e" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44087 } } peers { permanent_uuid: "74b028f8dcd8443493a06be381ff1317" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 44825 } } peers { permanent_uuid: "25f72b14a05d4ec6861e7a7bbbac9ef5" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 38397 } }
I20260808 16:52:07.739630 11642 raft_consensus.cc:374] T 3b3a66cee0b545dd9694e383f2069f6f P 74b028f8dcd8443493a06be381ff1317 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:52:07.739652 11642 raft_consensus.cc:729] T 3b3a66cee0b545dd9694e383f2069f6f P 74b028f8dcd8443493a06be381ff1317 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74b028f8dcd8443493a06be381ff1317, State: Initialized, Role: FOLLOWER
I20260808 16:52:07.739688 11642 consensus_queue.cc:260] T 3b3a66cee0b545dd9694e383f2069f6f P 74b028f8dcd8443493a06be381ff1317 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "982252a2a0f04384a24931f2215efa1e" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44087 } } peers { permanent_uuid: "74b028f8dcd8443493a06be381ff1317" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 44825 } } peers { permanent_uuid: "25f72b14a05d4ec6861e7a7bbbac9ef5" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 38397 } }
I20260808 16:52:07.739774 11642 ts_tablet_manager.cc:1428] T 3b3a66cee0b545dd9694e383f2069f6f P 74b028f8dcd8443493a06be381ff1317: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:07.739820 11643 tablet_bootstrap.cc:654] T 3b3a66cee0b545dd9694e383f2069f6f P 982252a2a0f04384a24931f2215efa1e: Neither blocks nor log segments found. Creating new log.
I20260808 16:52:07.739943 11654 tablet_replica.cc:439] T 3b3a66cee0b545dd9694e383f2069f6f P 74b028f8dcd8443493a06be381ff1317: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "982252a2a0f04384a24931f2215efa1e" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44087 } } peers { permanent_uuid: "74b028f8dcd8443493a06be381ff1317" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 44825 } } peers { permanent_uuid: "25f72b14a05d4ec6861e7a7bbbac9ef5" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 38397 } } }
I20260808 16:52:07.740038 11654 tablet_replica.cc:442] T 3b3a66cee0b545dd9694e383f2069f6f P 74b028f8dcd8443493a06be381ff1317: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 16:52:07.740150 11651 tablet_replica.cc:439] T 3b3a66cee0b545dd9694e383f2069f6f P 25f72b14a05d4ec6861e7a7bbbac9ef5: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "982252a2a0f04384a24931f2215efa1e" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44087 } } peers { permanent_uuid: "74b028f8dcd8443493a06be381ff1317" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 44825 } } peers { permanent_uuid: "25f72b14a05d4ec6861e7a7bbbac9ef5" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 38397 } } }
I20260808 16:52:07.740245 11651 tablet_replica.cc:442] T 3b3a66cee0b545dd9694e383f2069f6f P 25f72b14a05d4ec6861e7a7bbbac9ef5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 16:52:07.740505 11643 tablet_bootstrap.cc:492] T 3b3a66cee0b545dd9694e383f2069f6f P 982252a2a0f04384a24931f2215efa1e: No bootstrap required, opened a new log
I20260808 16:52:07.740554 11643 ts_tablet_manager.cc:1397] T 3b3a66cee0b545dd9694e383f2069f6f P 982252a2a0f04384a24931f2215efa1e: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 16:52:07.740695 11643 raft_consensus.cc:348] T 3b3a66cee0b545dd9694e383f2069f6f P 982252a2a0f04384a24931f2215efa1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "982252a2a0f04384a24931f2215efa1e" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44087 } } peers { permanent_uuid: "74b028f8dcd8443493a06be381ff1317" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 44825 } } peers { permanent_uuid: "25f72b14a05d4ec6861e7a7bbbac9ef5" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 38397 } }
I20260808 16:52:07.740772 11643 raft_consensus.cc:374] T 3b3a66cee0b545dd9694e383f2069f6f P 982252a2a0f04384a24931f2215efa1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:52:07.740800 11643 raft_consensus.cc:729] T 3b3a66cee0b545dd9694e383f2069f6f P 982252a2a0f04384a24931f2215efa1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 982252a2a0f04384a24931f2215efa1e, State: Initialized, Role: FOLLOWER
I20260808 16:52:07.740850 11643 consensus_queue.cc:260] T 3b3a66cee0b545dd9694e383f2069f6f P 982252a2a0f04384a24931f2215efa1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "982252a2a0f04384a24931f2215efa1e" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44087 } } peers { permanent_uuid: "74b028f8dcd8443493a06be381ff1317" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 44825 } } peers { permanent_uuid: "25f72b14a05d4ec6861e7a7bbbac9ef5" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 38397 } }
I20260808 16:52:07.740963 11643 ts_tablet_manager.cc:1428] T 3b3a66cee0b545dd9694e383f2069f6f P 982252a2a0f04384a24931f2215efa1e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:07.741021 11648 tablet_replica.cc:439] T 3b3a66cee0b545dd9694e383f2069f6f P 982252a2a0f04384a24931f2215efa1e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "982252a2a0f04384a24931f2215efa1e" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44087 } } peers { permanent_uuid: "74b028f8dcd8443493a06be381ff1317" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 44825 } } peers { permanent_uuid: "25f72b14a05d4ec6861e7a7bbbac9ef5" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 38397 } } }
I20260808 16:52:07.741106 11648 tablet_replica.cc:442] T 3b3a66cee0b545dd9694e383f2069f6f P 982252a2a0f04384a24931f2215efa1e: This TxnStatusTablet replica's current role is: FOLLOWER
W20260808 16:52:07.745985 11609 tablet.cc:2369] T 3b3a66cee0b545dd9694e383f2069f6f P 25f72b14a05d4ec6861e7a7bbbac9ef5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 16:52:07.884557 11648 raft_consensus.cc:482] T 3b3a66cee0b545dd9694e383f2069f6f P 982252a2a0f04384a24931f2215efa1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 16:52:07.884654 11648 raft_consensus.cc:504] T 3b3a66cee0b545dd9694e383f2069f6f P 982252a2a0f04384a24931f2215efa1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "982252a2a0f04384a24931f2215efa1e" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44087 } } peers { permanent_uuid: "74b028f8dcd8443493a06be381ff1317" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 44825 } } peers { permanent_uuid: "25f72b14a05d4ec6861e7a7bbbac9ef5" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 38397 } }
I20260808 16:52:07.884970 11648 leader_election.cc:288] T 3b3a66cee0b545dd9694e383f2069f6f P 982252a2a0f04384a24931f2215efa1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 74b028f8dcd8443493a06be381ff1317 (127.8.24.2:44825), 25f72b14a05d4ec6861e7a7bbbac9ef5 (127.8.24.3:38397)
I20260808 16:52:07.887841 11641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:07.885239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:35093 (local address 127.8.24.3:38397)
0808 16:52:07.885280 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:07.885282 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:07.885321 (+ 39us) server_negotiation.cc:407] Connection header received
0808 16:52:07.885398 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:07.885399 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:07.885408 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:07.885454 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:52:07.885601 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:07.886054 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:07.886996 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:07.887518 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:07.887566 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:07.887575 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:07.887768 (+ 193us) server_negotiation.cc:299] Negotiation successful
0808 16:52:07.887792 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 16:52:07.887893 11674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:07.885146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.3:38397 (local address 127.8.24.1:35093)
0808 16:52:07.885262 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:07.885271 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:07.885331 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:07.885465 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:07.885466 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:07.885468 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:52:07.885539 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:07.885541 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:07.886121 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:07.886122 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:07.886917 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:07.886920 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:07.887623 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:07.887624 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:07.887697 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:07.887702 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 16:52:07.887764 (+ 62us) client_negotiation.cc:240] Negotiation successful
0808 16:52:07.887801 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":55,"threads_started":1}
I20260808 16:52:07.888197 11534 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b3a66cee0b545dd9694e383f2069f6f" candidate_uuid: "982252a2a0f04384a24931f2215efa1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25f72b14a05d4ec6861e7a7bbbac9ef5" is_pre_election: true
I20260808 16:52:07.888326 11673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:07.885116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.2:44825 (local address 127.8.24.1:37421)
0808 16:52:07.885251 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:07.885261 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:07.885328 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:07.885454 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:07.885454 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:07.885457 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:52:07.885530 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:07.885532 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:07.886137 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:07.886137 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:07.887367 (+ 1230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:07.887371 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:07.888080 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:07.888081 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:07.888147 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:07.888150 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 16:52:07.888221 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 16:52:07.888256 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":62,"threads_started":1}
I20260808 16:52:07.888325 11534 raft_consensus.cc:2454] T 3b3a66cee0b545dd9694e383f2069f6f P 25f72b14a05d4ec6861e7a7bbbac9ef5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 982252a2a0f04384a24931f2215efa1e in term 0.
I20260808 16:52:07.888331 11635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:07.885189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:37421 (local address 127.8.24.2:44825)
0808 16:52:07.885244 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:07.885246 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:07.885325 (+ 79us) server_negotiation.cc:407] Connection header received
0808 16:52:07.885392 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:07.885393 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:07.885402 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:07.885439 (+ 37us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:52:07.885590 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:07.886065 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:07.887463 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:07.887973 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:07.888038 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:07.888048 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:07.888232 (+ 184us) server_negotiation.cc:299] Negotiation successful
0808 16:52:07.888271 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 16:52:07.888523 10995 leader_election.cc:302] T 3b3a66cee0b545dd9694e383f2069f6f P 982252a2a0f04384a24931f2215efa1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25f72b14a05d4ec6861e7a7bbbac9ef5, 982252a2a0f04384a24931f2215efa1e; no voters:
I20260808 16:52:07.888697 11648 raft_consensus.cc:2781] T 3b3a66cee0b545dd9694e383f2069f6f P 982252a2a0f04384a24931f2215efa1e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 16:52:07.888685 11309 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b3a66cee0b545dd9694e383f2069f6f" candidate_uuid: "982252a2a0f04384a24931f2215efa1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74b028f8dcd8443493a06be381ff1317" is_pre_election: true
I20260808 16:52:07.888741 11648 raft_consensus.cc:482] T 3b3a66cee0b545dd9694e383f2069f6f P 982252a2a0f04384a24931f2215efa1e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 16:52:07.888760 11648 raft_consensus.cc:3037] T 3b3a66cee0b545dd9694e383f2069f6f P 982252a2a0f04384a24931f2215efa1e [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:52:07.888785 11309 raft_consensus.cc:2454] T 3b3a66cee0b545dd9694e383f2069f6f P 74b028f8dcd8443493a06be381ff1317 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 982252a2a0f04384a24931f2215efa1e in term 0.
I20260808 16:52:07.889487 11648 raft_consensus.cc:504] T 3b3a66cee0b545dd9694e383f2069f6f P 982252a2a0f04384a24931f2215efa1e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "982252a2a0f04384a24931f2215efa1e" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44087 } } peers { permanent_uuid: "74b028f8dcd8443493a06be381ff1317" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 44825 } } peers { permanent_uuid: "25f72b14a05d4ec6861e7a7bbbac9ef5" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 38397 } }
I20260808 16:52:07.889600 11648 leader_election.cc:288] T 3b3a66cee0b545dd9694e383f2069f6f P 982252a2a0f04384a24931f2215efa1e [CANDIDATE]: Term 1 election: Requested vote from peers 74b028f8dcd8443493a06be381ff1317 (127.8.24.2:44825), 25f72b14a05d4ec6861e7a7bbbac9ef5 (127.8.24.3:38397)
I20260808 16:52:07.889757 11534 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b3a66cee0b545dd9694e383f2069f6f" candidate_uuid: "982252a2a0f04384a24931f2215efa1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25f72b14a05d4ec6861e7a7bbbac9ef5"
I20260808 16:52:07.889770 11309 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b3a66cee0b545dd9694e383f2069f6f" candidate_uuid: "982252a2a0f04384a24931f2215efa1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74b028f8dcd8443493a06be381ff1317"
I20260808 16:52:07.889839 11309 raft_consensus.cc:3037] T 3b3a66cee0b545dd9694e383f2069f6f P 74b028f8dcd8443493a06be381ff1317 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:52:07.889839 11534 raft_consensus.cc:3037] T 3b3a66cee0b545dd9694e383f2069f6f P 25f72b14a05d4ec6861e7a7bbbac9ef5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:52:07.890583 11309 raft_consensus.cc:2454] T 3b3a66cee0b545dd9694e383f2069f6f P 74b028f8dcd8443493a06be381ff1317 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 982252a2a0f04384a24931f2215efa1e in term 1.
I20260808 16:52:07.890643 11534 raft_consensus.cc:2454] T 3b3a66cee0b545dd9694e383f2069f6f P 25f72b14a05d4ec6861e7a7bbbac9ef5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 982252a2a0f04384a24931f2215efa1e in term 1.
I20260808 16:52:07.890779 10992 leader_election.cc:302] T 3b3a66cee0b545dd9694e383f2069f6f P 982252a2a0f04384a24931f2215efa1e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74b028f8dcd8443493a06be381ff1317, 982252a2a0f04384a24931f2215efa1e; no voters:
I20260808 16:52:07.890880 11648 raft_consensus.cc:2781] T 3b3a66cee0b545dd9694e383f2069f6f P 982252a2a0f04384a24931f2215efa1e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:52:07.890985 11648 raft_consensus.cc:686] T 3b3a66cee0b545dd9694e383f2069f6f P 982252a2a0f04384a24931f2215efa1e [term 1 LEADER]: Becoming Leader. State: Replica: 982252a2a0f04384a24931f2215efa1e, State: Running, Role: LEADER
I20260808 16:52:07.891075 11648 consensus_queue.cc:237] T 3b3a66cee0b545dd9694e383f2069f6f P 982252a2a0f04384a24931f2215efa1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "982252a2a0f04384a24931f2215efa1e" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44087 } } peers { permanent_uuid: "74b028f8dcd8443493a06be381ff1317" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 44825 } } peers { permanent_uuid: "25f72b14a05d4ec6861e7a7bbbac9ef5" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 38397 } }
I20260808 16:52:07.891506 11675 tablet_replica.cc:439] T 3b3a66cee0b545dd9694e383f2069f6f P 982252a2a0f04384a24931f2215efa1e: TxnStatusTablet state changed. Reason: New leader 982252a2a0f04384a24931f2215efa1e. Latest consensus state: current_term: 1 leader_uuid: "982252a2a0f04384a24931f2215efa1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "982252a2a0f04384a24931f2215efa1e" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44087 } } peers { permanent_uuid: "74b028f8dcd8443493a06be381ff1317" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 44825 } } peers { permanent_uuid: "25f72b14a05d4ec6861e7a7bbbac9ef5" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 38397 } } }
I20260808 16:52:07.891595 11675 tablet_replica.cc:442] T 3b3a66cee0b545dd9694e383f2069f6f P 982252a2a0f04384a24931f2215efa1e: This TxnStatusTablet replica's current role is: LEADER
I20260808 16:52:07.891793 11678 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 16:52:07.891779 10856 catalog_manager.cc:5495] T 3b3a66cee0b545dd9694e383f2069f6f P 982252a2a0f04384a24931f2215efa1e reported cstate change: term changed from 0 to 1, leader changed from <none> to 982252a2a0f04384a24931f2215efa1e (127.8.24.1). New cstate: current_term: 1 leader_uuid: "982252a2a0f04384a24931f2215efa1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "982252a2a0f04384a24931f2215efa1e" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44087 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74b028f8dcd8443493a06be381ff1317" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 44825 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25f72b14a05d4ec6861e7a7bbbac9ef5" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 38397 } health_report { overall_health: UNKNOWN } } }
I20260808 16:52:07.891928 11678 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 16:52:07.920214 11669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:07.917542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:44087 (local address 127.0.0.1:43156)
0808 16:52:07.917621 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:07.917632 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:07.917679 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:07.917827 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:07.917828 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:07.917829 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 16:52:07.917871 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:07.917873 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:07.918701 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:07.918701 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:07.919326 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:07.919330 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:07.919756 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:07.919757 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:07.919809 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:07.919823 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 16:52:07.920067 (+ 244us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 16:52:07.920070 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 16:52:07.920097 (+ 27us) client_negotiation.cc:240] Negotiation successful
0808 16:52:07.920145 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260808 16:52:07.920217 11638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:07.917601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43156 (local address 127.8.24.1:44087)
0808 16:52:07.917647 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:07.917649 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:07.917663 (+ 14us) server_negotiation.cc:407] Connection header received
0808 16:52:07.917737 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:07.917738 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:07.917756 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:07.917822 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 16:52:07.917919 (+ 97us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:07.918637 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:07.919395 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:07.919686 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:07.919750 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:07.919898 (+ 148us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 16:52:07.920014 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 16:52:07.920087 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:07.920104 (+ 17us) server_negotiation.cc:299] Negotiation successful
0808 16:52:07.920142 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 16:52:07.922148 11534 raft_consensus.cc:1261] T 3b3a66cee0b545dd9694e383f2069f6f P 25f72b14a05d4ec6861e7a7bbbac9ef5 [term 1 FOLLOWER]: Refusing update from remote peer 982252a2a0f04384a24931f2215efa1e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:52:07.922171 11309 raft_consensus.cc:1261] T 3b3a66cee0b545dd9694e383f2069f6f P 74b028f8dcd8443493a06be381ff1317 [term 1 FOLLOWER]: Refusing update from remote peer 982252a2a0f04384a24931f2215efa1e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:52:07.922448 11675 consensus_queue.cc:1035] T 3b3a66cee0b545dd9694e383f2069f6f P 982252a2a0f04384a24931f2215efa1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "25f72b14a05d4ec6861e7a7bbbac9ef5" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 38397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:52:07.922575 11648 consensus_queue.cc:1035] T 3b3a66cee0b545dd9694e383f2069f6f P 982252a2a0f04384a24931f2215efa1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "74b028f8dcd8443493a06be381ff1317" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 44825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:52:07.923677 11651 tablet_replica.cc:439] T 3b3a66cee0b545dd9694e383f2069f6f P 25f72b14a05d4ec6861e7a7bbbac9ef5: TxnStatusTablet state changed. Reason: New leader 982252a2a0f04384a24931f2215efa1e. Latest consensus state: current_term: 1 leader_uuid: "982252a2a0f04384a24931f2215efa1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "982252a2a0f04384a24931f2215efa1e" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44087 } } peers { permanent_uuid: "74b028f8dcd8443493a06be381ff1317" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 44825 } } peers { permanent_uuid: "25f72b14a05d4ec6861e7a7bbbac9ef5" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 38397 } } }
I20260808 16:52:07.923698 11654 tablet_replica.cc:439] T 3b3a66cee0b545dd9694e383f2069f6f P 74b028f8dcd8443493a06be381ff1317: TxnStatusTablet state changed. Reason: New leader 982252a2a0f04384a24931f2215efa1e. Latest consensus state: current_term: 1 leader_uuid: "982252a2a0f04384a24931f2215efa1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "982252a2a0f04384a24931f2215efa1e" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44087 } } peers { permanent_uuid: "74b028f8dcd8443493a06be381ff1317" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 44825 } } peers { permanent_uuid: "25f72b14a05d4ec6861e7a7bbbac9ef5" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 38397 } } }
I20260808 16:52:07.923763 11651 tablet_replica.cc:442] T 3b3a66cee0b545dd9694e383f2069f6f P 25f72b14a05d4ec6861e7a7bbbac9ef5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 16:52:07.923781 11654 tablet_replica.cc:442] T 3b3a66cee0b545dd9694e383f2069f6f P 74b028f8dcd8443493a06be381ff1317: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 16:52:07.924731 11648 tablet_replica.cc:439] T 3b3a66cee0b545dd9694e383f2069f6f P 982252a2a0f04384a24931f2215efa1e: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "982252a2a0f04384a24931f2215efa1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "982252a2a0f04384a24931f2215efa1e" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44087 } } peers { permanent_uuid: "74b028f8dcd8443493a06be381ff1317" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 44825 } } peers { permanent_uuid: "25f72b14a05d4ec6861e7a7bbbac9ef5" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 38397 } } }
I20260808 16:52:07.924805 11648 tablet_replica.cc:442] T 3b3a66cee0b545dd9694e383f2069f6f P 982252a2a0f04384a24931f2215efa1e: This TxnStatusTablet replica's current role is: LEADER
I20260808 16:52:07.924847 11648 tablet_replica.cc:439] T 3b3a66cee0b545dd9694e383f2069f6f P 982252a2a0f04384a24931f2215efa1e: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "982252a2a0f04384a24931f2215efa1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "982252a2a0f04384a24931f2215efa1e" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44087 } } peers { permanent_uuid: "74b028f8dcd8443493a06be381ff1317" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 44825 } } peers { permanent_uuid: "25f72b14a05d4ec6861e7a7bbbac9ef5" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 38397 } } }
I20260808 16:52:07.924881 11648 tablet_replica.cc:442] T 3b3a66cee0b545dd9694e383f2069f6f P 982252a2a0f04384a24931f2215efa1e: This TxnStatusTablet replica's current role is: LEADER
I20260808 16:52:07.924906 11651 tablet_replica.cc:439] T 3b3a66cee0b545dd9694e383f2069f6f P 25f72b14a05d4ec6861e7a7bbbac9ef5: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "982252a2a0f04384a24931f2215efa1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "982252a2a0f04384a24931f2215efa1e" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44087 } } peers { permanent_uuid: "74b028f8dcd8443493a06be381ff1317" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 44825 } } peers { permanent_uuid: "25f72b14a05d4ec6861e7a7bbbac9ef5" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 38397 } } }
I20260808 16:52:07.924943 11682 mvcc.cc:205] Tried to move back new op lower bound from 7316307672766951424 to 7316307672642355200. Current Snapshot: MvccSnapshot[applied={T|T < 7316307672766951424}]
I20260808 16:52:07.924924 11654 tablet_replica.cc:439] T 3b3a66cee0b545dd9694e383f2069f6f P 74b028f8dcd8443493a06be381ff1317: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "982252a2a0f04384a24931f2215efa1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "982252a2a0f04384a24931f2215efa1e" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44087 } } peers { permanent_uuid: "74b028f8dcd8443493a06be381ff1317" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 44825 } } peers { permanent_uuid: "25f72b14a05d4ec6861e7a7bbbac9ef5" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 38397 } } }
I20260808 16:52:07.924983 11651 tablet_replica.cc:442] T 3b3a66cee0b545dd9694e383f2069f6f P 25f72b14a05d4ec6861e7a7bbbac9ef5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 16:52:07.924988 11654 tablet_replica.cc:442] T 3b3a66cee0b545dd9694e383f2069f6f P 74b028f8dcd8443493a06be381ff1317: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 16:52:07.925479 11684 mvcc.cc:205] Tried to move back new op lower bound from 7316307672766951424 to 7316307672642355200. Current Snapshot: MvccSnapshot[applied={T|T < 7316307672766951424}]
I20260808 16:52:07.925753 11683 mvcc.cc:205] Tried to move back new op lower bound from 7316307672766951424 to 7316307672642355200. Current Snapshot: MvccSnapshot[applied={T|T < 7316307672766951424}]
I20260808 16:52:07.929744 11641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:07.927664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50980 (local address 127.8.24.3:38397)
0808 16:52:07.927697 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:07.927699 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:07.927710 (+ 11us) server_negotiation.cc:407] Connection header received
0808 16:52:07.927774 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:07.927775 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:07.927787 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:07.927846 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 16:52:07.927971 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:07.928400 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:07.929086 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:07.929283 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:07.929332 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:07.929476 (+ 144us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 16:52:07.929560 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 16:52:07.929621 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:07.929657 (+ 36us) server_negotiation.cc:299] Negotiation successful
0808 16:52:07.929678 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260808 16:52:07.933905 11638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:07.931932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:35901 (local address 127.8.24.1:44087)
0808 16:52:07.931968 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:07.931970 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:07.931980 (+ 10us) server_negotiation.cc:407] Connection header received
0808 16:52:07.932041 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:07.932042 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:07.932050 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:07.932093 (+ 43us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 16:52:07.932186 (+ 93us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:07.932612 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:07.933310 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:07.933498 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:07.933549 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:07.933709 (+ 160us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 16:52:07.933739 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 16:52:07.933780 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:07.933833 (+ 53us) server_negotiation.cc:299] Negotiation successful
0808 16:52:07.933848 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 16:52:07.933926 11619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:07.931876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:44087 (local address 127.8.24.3:35901)
0808 16:52:07.931945 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:07.931954 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:07.931990 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:07.932095 (+ 105us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:07.932096 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:07.932097 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 16:52:07.932143 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:07.932145 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:07.932685 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:07.932686 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:07.933241 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:07.933243 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:07.933562 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:07.933563 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:07.933623 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:07.933635 (+ 12us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 16:52:07.933791 (+ 156us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 16:52:07.933793 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 16:52:07.933831 (+ 38us) client_negotiation.cc:240] Negotiation successful
0808 16:52:07.933867 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260808 16:52:07.936143 11309 raft_consensus.cc:1261] T fbcb3fc43b2d4448b28a5b7b7e4b4f65 P 74b028f8dcd8443493a06be381ff1317 [term 1 FOLLOWER]: Refusing update from remote peer 25f72b14a05d4ec6861e7a7bbbac9ef5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:52:07.936195 11092 raft_consensus.cc:1261] T fbcb3fc43b2d4448b28a5b7b7e4b4f65 P 982252a2a0f04384a24931f2215efa1e [term 1 FOLLOWER]: Refusing update from remote peer 25f72b14a05d4ec6861e7a7bbbac9ef5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:52:07.936439 11656 consensus_queue.cc:1035] T fbcb3fc43b2d4448b28a5b7b7e4b4f65 P 25f72b14a05d4ec6861e7a7bbbac9ef5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74b028f8dcd8443493a06be381ff1317" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 44825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:52:07.936532 11651 consensus_queue.cc:1035] T fbcb3fc43b2d4448b28a5b7b7e4b4f65 P 25f72b14a05d4ec6861e7a7bbbac9ef5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "982252a2a0f04384a24931f2215efa1e" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:52:07.944751 11641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:07.942155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:55705 (local address 127.8.24.3:38397)
0808 16:52:07.942189 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:07.942190 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:07.942198 (+ 8us) server_negotiation.cc:407] Connection header received
0808 16:52:07.942217 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:07.942217 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:07.942227 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:07.942275 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 16:52:07.942513 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:07.942927 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:07.944020 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:07.944321 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:07.944358 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:07.944557 (+ 199us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 16:52:07.944582 (+ 25us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 16:52:07.944620 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:07.944687 (+ 67us) server_negotiation.cc:299] Negotiation successful
0808 16:52:07.944701 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 16:52:07.944814 11705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:07.941956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.3:38397 (local address 127.8.24.1:55705)
0808 16:52:07.942103 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:07.942113 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:07.942161 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:07.942279 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:07.942279 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:07.942383 (+ 104us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 16:52:07.942452 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:07.942454 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:07.942993 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:07.942994 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:07.943921 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:07.943924 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:07.944409 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:07.944409 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:07.944471 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:07.944485 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 16:52:07.944639 (+ 154us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 16:52:07.944640 (+ 1us) client_negotiation.cc:769] Sending connection context
0808 16:52:07.944683 (+ 43us) client_negotiation.cc:240] Negotiation successful
0808 16:52:07.944720 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":80,"threads_started":1}
I20260808 16:52:07.954280 10856 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59230:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 16:52:07.954406 10856 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59230:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 16:52:07.955082 10856 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5627ffa85f6546a1a64643aa34ab1e6f: Sending DeleteTablet for 3 replicas of tablet fbcb3fc43b2d4448b28a5b7b7e4b4f65
I20260808 16:52:07.955469 11278 tablet_service.cc:1514] Processing DeleteTablet for tablet fbcb3fc43b2d4448b28a5b7b7e4b4f65 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:52:07 UTC) from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:51682
I20260808 16:52:07.955675 11501 tablet_service.cc:1514] Processing DeleteTablet for tablet fbcb3fc43b2d4448b28a5b7b7e4b4f65 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:52:07 UTC) from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:50978
I20260808 16:52:07.955701 11711 tablet_replica.cc:330] T fbcb3fc43b2d4448b28a5b7b7e4b4f65 P 74b028f8dcd8443493a06be381ff1317: stopping tablet replica
I20260808 16:52:07.955804 11711 raft_consensus.cc:2229] T fbcb3fc43b2d4448b28a5b7b7e4b4f65 P 74b028f8dcd8443493a06be381ff1317 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 16:52:07.955854 11711 raft_consensus.cc:2258] T fbcb3fc43b2d4448b28a5b7b7e4b4f65 P 74b028f8dcd8443493a06be381ff1317 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:52:07.956027 11060 tablet_service.cc:1514] Processing DeleteTablet for tablet fbcb3fc43b2d4448b28a5b7b7e4b4f65 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:52:07 UTC) from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:43148
I20260808 16:52:07.956305 11711 ts_tablet_manager.cc:1905] T fbcb3fc43b2d4448b28a5b7b7e4b4f65 P 74b028f8dcd8443493a06be381ff1317: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:52:07.956269 10856 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59230:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 16:52:07.956406 10856 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 16:52:07.957749 11501 tablet_service.cc:1467] Processing CreateTablet for tablet 7df36578fae3427c8f1e5b5927621a39 (DEFAULT_TABLE table=test-table [id=8d8eefa06a764f0099927da857db17a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:52:07.957875 11501 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7df36578fae3427c8f1e5b5927621a39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:52:07.958611 11711 ts_tablet_manager.cc:1918] T fbcb3fc43b2d4448b28a5b7b7e4b4f65 P 74b028f8dcd8443493a06be381ff1317: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 16:52:07.958673 11711 log.cc:1196] T fbcb3fc43b2d4448b28a5b7b7e4b4f65 P 74b028f8dcd8443493a06be381ff1317: Deleting WAL directory at /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/minicluster-data/ts-1/wal/wals/fbcb3fc43b2d4448b28a5b7b7e4b4f65
I20260808 16:52:07.959231 11711 ts_tablet_manager.cc:1939] T fbcb3fc43b2d4448b28a5b7b7e4b4f65 P 74b028f8dcd8443493a06be381ff1317: Deleting consensus metadata
I20260808 16:52:07.959244 11645 tablet_bootstrap.cc:492] T 7df36578fae3427c8f1e5b5927621a39 P 25f72b14a05d4ec6861e7a7bbbac9ef5: Bootstrap starting.
I20260808 16:52:07.959645 11645 tablet_bootstrap.cc:654] T 7df36578fae3427c8f1e5b5927621a39 P 25f72b14a05d4ec6861e7a7bbbac9ef5: Neither blocks nor log segments found. Creating new log.
I20260808 16:52:07.959637 10837 catalog_manager.cc:4841] TS 74b028f8dcd8443493a06be381ff1317 (127.8.24.2:44825): tablet fbcb3fc43b2d4448b28a5b7b7e4b4f65 (table test-table [id=4c0d4216b57b44dd9ce83624024571db]) successfully deleted
I20260808 16:52:07.960273 11278 tablet_service.cc:1467] Processing CreateTablet for tablet 7df36578fae3427c8f1e5b5927621a39 (DEFAULT_TABLE table=test-table [id=8d8eefa06a764f0099927da857db17a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:52:07.960391 11278 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7df36578fae3427c8f1e5b5927621a39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:52:07.961503 11642 tablet_bootstrap.cc:492] T 7df36578fae3427c8f1e5b5927621a39 P 74b028f8dcd8443493a06be381ff1317: Bootstrap starting.
I20260808 16:52:07.961746 11060 tablet_service.cc:1467] Processing CreateTablet for tablet 7df36578fae3427c8f1e5b5927621a39 (DEFAULT_TABLE table=test-table [id=8d8eefa06a764f0099927da857db17a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:52:07.961848 11060 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7df36578fae3427c8f1e5b5927621a39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:52:07.961861 11642 tablet_bootstrap.cc:654] T 7df36578fae3427c8f1e5b5927621a39 P 74b028f8dcd8443493a06be381ff1317: Neither blocks nor log segments found. Creating new log.
I20260808 16:52:07.962672 11642 tablet_bootstrap.cc:492] T 7df36578fae3427c8f1e5b5927621a39 P 74b028f8dcd8443493a06be381ff1317: No bootstrap required, opened a new log
I20260808 16:52:07.962711 11642 ts_tablet_manager.cc:1397] T 7df36578fae3427c8f1e5b5927621a39 P 74b028f8dcd8443493a06be381ff1317: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260808 16:52:07.962847 11642 raft_consensus.cc:348] T 7df36578fae3427c8f1e5b5927621a39 P 74b028f8dcd8443493a06be381ff1317 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "982252a2a0f04384a24931f2215efa1e" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44087 } } peers { permanent_uuid: "74b028f8dcd8443493a06be381ff1317" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 44825 } } peers { permanent_uuid: "25f72b14a05d4ec6861e7a7bbbac9ef5" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 38397 } }
I20260808 16:52:07.962921 11642 raft_consensus.cc:374] T 7df36578fae3427c8f1e5b5927621a39 P 74b028f8dcd8443493a06be381ff1317 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:52:07.962940 11642 raft_consensus.cc:729] T 7df36578fae3427c8f1e5b5927621a39 P 74b028f8dcd8443493a06be381ff1317 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74b028f8dcd8443493a06be381ff1317, State: Initialized, Role: FOLLOWER
I20260808 16:52:07.962975 11642 consensus_queue.cc:260] T 7df36578fae3427c8f1e5b5927621a39 P 74b028f8dcd8443493a06be381ff1317 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "982252a2a0f04384a24931f2215efa1e" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44087 } } peers { permanent_uuid: "74b028f8dcd8443493a06be381ff1317" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 44825 } } peers { permanent_uuid: "25f72b14a05d4ec6861e7a7bbbac9ef5" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 38397 } }
I20260808 16:52:07.963024 11643 tablet_bootstrap.cc:492] T 7df36578fae3427c8f1e5b5927621a39 P 982252a2a0f04384a24931f2215efa1e: Bootstrap starting.
I20260808 16:52:07.963045 11642 ts_tablet_manager.cc:1428] T 7df36578fae3427c8f1e5b5927621a39 P 74b028f8dcd8443493a06be381ff1317: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:07.963382 11643 tablet_bootstrap.cc:654] T 7df36578fae3427c8f1e5b5927621a39 P 982252a2a0f04384a24931f2215efa1e: Neither blocks nor log segments found. Creating new log.
I20260808 16:52:07.964187 11643 tablet_bootstrap.cc:492] T 7df36578fae3427c8f1e5b5927621a39 P 982252a2a0f04384a24931f2215efa1e: No bootstrap required, opened a new log
I20260808 16:52:07.964234 11643 ts_tablet_manager.cc:1397] T 7df36578fae3427c8f1e5b5927621a39 P 982252a2a0f04384a24931f2215efa1e: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 16:52:07.964285 11645 tablet_bootstrap.cc:492] T 7df36578fae3427c8f1e5b5927621a39 P 25f72b14a05d4ec6861e7a7bbbac9ef5: No bootstrap required, opened a new log
I20260808 16:52:07.964337 11645 ts_tablet_manager.cc:1397] T 7df36578fae3427c8f1e5b5927621a39 P 25f72b14a05d4ec6861e7a7bbbac9ef5: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20260808 16:52:07.964354 11643 raft_consensus.cc:348] T 7df36578fae3427c8f1e5b5927621a39 P 982252a2a0f04384a24931f2215efa1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "982252a2a0f04384a24931f2215efa1e" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44087 } } peers { permanent_uuid: "74b028f8dcd8443493a06be381ff1317" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 44825 } } peers { permanent_uuid: "25f72b14a05d4ec6861e7a7bbbac9ef5" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 38397 } }
I20260808 16:52:07.964409 11643 raft_consensus.cc:374] T 7df36578fae3427c8f1e5b5927621a39 P 982252a2a0f04384a24931f2215efa1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:52:07.964427 11643 raft_consensus.cc:729] T 7df36578fae3427c8f1e5b5927621a39 P 982252a2a0f04384a24931f2215efa1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 982252a2a0f04384a24931f2215efa1e, State: Initialized, Role: FOLLOWER
I20260808 16:52:07.964464 11643 consensus_queue.cc:260] T 7df36578fae3427c8f1e5b5927621a39 P 982252a2a0f04384a24931f2215efa1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "982252a2a0f04384a24931f2215efa1e" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44087 } } peers { permanent_uuid: "74b028f8dcd8443493a06be381ff1317" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 44825 } } peers { permanent_uuid: "25f72b14a05d4ec6861e7a7bbbac9ef5" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 38397 } }
I20260808 16:52:07.964468 11645 raft_consensus.cc:348] T 7df36578fae3427c8f1e5b5927621a39 P 25f72b14a05d4ec6861e7a7bbbac9ef5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "982252a2a0f04384a24931f2215efa1e" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44087 } } peers { permanent_uuid: "74b028f8dcd8443493a06be381ff1317" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 44825 } } peers { permanent_uuid: "25f72b14a05d4ec6861e7a7bbbac9ef5" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 38397 } }
I20260808 16:52:07.964524 11645 raft_consensus.cc:374] T 7df36578fae3427c8f1e5b5927621a39 P 25f72b14a05d4ec6861e7a7bbbac9ef5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:52:07.964534 11643 ts_tablet_manager.cc:1428] T 7df36578fae3427c8f1e5b5927621a39 P 982252a2a0f04384a24931f2215efa1e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:07.964541 11645 raft_consensus.cc:729] T 7df36578fae3427c8f1e5b5927621a39 P 25f72b14a05d4ec6861e7a7bbbac9ef5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25f72b14a05d4ec6861e7a7bbbac9ef5, State: Initialized, Role: FOLLOWER
I20260808 16:52:07.964579 11645 consensus_queue.cc:260] T 7df36578fae3427c8f1e5b5927621a39 P 25f72b14a05d4ec6861e7a7bbbac9ef5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "982252a2a0f04384a24931f2215efa1e" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44087 } } peers { permanent_uuid: "74b028f8dcd8443493a06be381ff1317" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 44825 } } peers { permanent_uuid: "25f72b14a05d4ec6861e7a7bbbac9ef5" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 38397 } }
I20260808 16:52:07.964654 11645 ts_tablet_manager.cc:1428] T 7df36578fae3427c8f1e5b5927621a39 P 25f72b14a05d4ec6861e7a7bbbac9ef5: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:07.964871 11713 tablet_replica.cc:330] T fbcb3fc43b2d4448b28a5b7b7e4b4f65 P 982252a2a0f04384a24931f2215efa1e: stopping tablet replica
I20260808 16:52:07.965006 11713 raft_consensus.cc:2229] T fbcb3fc43b2d4448b28a5b7b7e4b4f65 P 982252a2a0f04384a24931f2215efa1e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 16:52:07.965103 11713 raft_consensus.cc:2258] T fbcb3fc43b2d4448b28a5b7b7e4b4f65 P 982252a2a0f04384a24931f2215efa1e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:52:07.965364 11712 tablet_replica.cc:330] T fbcb3fc43b2d4448b28a5b7b7e4b4f65 P 25f72b14a05d4ec6861e7a7bbbac9ef5: stopping tablet replica
I20260808 16:52:07.965456 11712 raft_consensus.cc:2229] T fbcb3fc43b2d4448b28a5b7b7e4b4f65 P 25f72b14a05d4ec6861e7a7bbbac9ef5 [term 1 LEADER]: Raft consensus shutting down.
I20260808 16:52:07.965545 11712 raft_consensus.cc:2258] T fbcb3fc43b2d4448b28a5b7b7e4b4f65 P 25f72b14a05d4ec6861e7a7bbbac9ef5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:52:07.965811 11713 ts_tablet_manager.cc:1905] T fbcb3fc43b2d4448b28a5b7b7e4b4f65 P 982252a2a0f04384a24931f2215efa1e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:52:07.965941 11712 ts_tablet_manager.cc:1905] T fbcb3fc43b2d4448b28a5b7b7e4b4f65 P 25f72b14a05d4ec6861e7a7bbbac9ef5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:52:07.967314 11713 ts_tablet_manager.cc:1918] T fbcb3fc43b2d4448b28a5b7b7e4b4f65 P 982252a2a0f04384a24931f2215efa1e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 16:52:07.967433 11713 log.cc:1196] T fbcb3fc43b2d4448b28a5b7b7e4b4f65 P 982252a2a0f04384a24931f2215efa1e: Deleting WAL directory at /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/minicluster-data/ts-0/wal/wals/fbcb3fc43b2d4448b28a5b7b7e4b4f65
I20260808 16:52:07.967741 11712 ts_tablet_manager.cc:1918] T fbcb3fc43b2d4448b28a5b7b7e4b4f65 P 25f72b14a05d4ec6861e7a7bbbac9ef5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 16:52:07.967808 11713 ts_tablet_manager.cc:1939] T fbcb3fc43b2d4448b28a5b7b7e4b4f65 P 982252a2a0f04384a24931f2215efa1e: Deleting consensus metadata
I20260808 16:52:07.967861 11712 log.cc:1196] T fbcb3fc43b2d4448b28a5b7b7e4b4f65 P 25f72b14a05d4ec6861e7a7bbbac9ef5: Deleting WAL directory at /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/minicluster-data/ts-2/wal/wals/fbcb3fc43b2d4448b28a5b7b7e4b4f65
I20260808 16:52:07.968153 11712 ts_tablet_manager.cc:1939] T fbcb3fc43b2d4448b28a5b7b7e4b4f65 P 25f72b14a05d4ec6861e7a7bbbac9ef5: Deleting consensus metadata
I20260808 16:52:07.968237 10840 catalog_manager.cc:4841] TS 982252a2a0f04384a24931f2215efa1e (127.8.24.1:44087): tablet fbcb3fc43b2d4448b28a5b7b7e4b4f65 (table test-table [id=4c0d4216b57b44dd9ce83624024571db]) successfully deleted
I20260808 16:52:07.968532 10841 catalog_manager.cc:4841] TS 25f72b14a05d4ec6861e7a7bbbac9ef5 (127.8.24.3:38397): tablet fbcb3fc43b2d4448b28a5b7b7e4b4f65 (table test-table [id=4c0d4216b57b44dd9ce83624024571db]) successfully deleted
I20260808 16:52:08.169185 11654 raft_consensus.cc:482] T 7df36578fae3427c8f1e5b5927621a39 P 74b028f8dcd8443493a06be381ff1317 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 16:52:08.169272 11654 raft_consensus.cc:504] T 7df36578fae3427c8f1e5b5927621a39 P 74b028f8dcd8443493a06be381ff1317 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "982252a2a0f04384a24931f2215efa1e" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44087 } } peers { permanent_uuid: "74b028f8dcd8443493a06be381ff1317" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 44825 } } peers { permanent_uuid: "25f72b14a05d4ec6861e7a7bbbac9ef5" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 38397 } }
I20260808 16:52:08.169548 11654 leader_election.cc:288] T 7df36578fae3427c8f1e5b5927621a39 P 74b028f8dcd8443493a06be381ff1317 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 982252a2a0f04384a24931f2215efa1e (127.8.24.1:44087), 25f72b14a05d4ec6861e7a7bbbac9ef5 (127.8.24.3:38397)
I20260808 16:52:08.172295 11638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:08.169725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:34471 (local address 127.8.24.1:44087)
0808 16:52:08.169809 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:08.169812 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:08.169878 (+ 66us) server_negotiation.cc:407] Connection header received
0808 16:52:08.169926 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:08.169927 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:08.169936 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:08.169986 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:52:08.170129 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:08.170631 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:08.171644 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:08.171996 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:08.172056 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:08.172066 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:08.172196 (+ 130us) server_negotiation.cc:299] Negotiation successful
0808 16:52:08.172229 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 16:52:08.172286 11641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:08.169778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:33733 (local address 127.8.24.3:38397)
0808 16:52:08.169817 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:08.169819 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:08.169914 (+ 95us) server_negotiation.cc:407] Connection header received
0808 16:52:08.169967 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:08.169968 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:08.169977 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:08.170027 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:52:08.170156 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:08.170679 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:08.171641 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:08.171983 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:08.172037 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:08.172047 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:08.172188 (+ 141us) server_negotiation.cc:299] Negotiation successful
0808 16:52:08.172217 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 16:52:08.172326 11725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:08.169701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.3:38397 (local address 127.8.24.2:33733)
0808 16:52:08.169846 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:08.169854 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:08.169917 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:08.170036 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:08.170037 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:08.170039 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:52:08.170096 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:08.170098 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:08.170748 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:08.170749 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:08.171559 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:08.171561 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:08.172052 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:08.172053 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:08.172127 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:08.172131 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 16:52:08.172175 (+ 44us) client_negotiation.cc:240] Negotiation successful
0808 16:52:08.172216 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"mutex_wait_us":39,"thread_start_us":84,"threads_started":1}
I20260808 16:52:08.172333 11723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:08.169682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:44087 (local address 127.8.24.2:34471)
0808 16:52:08.169794 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:08.169815 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:08.169875 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:08.169988 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:08.169989 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:08.169991 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:52:08.170071 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:08.170073 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:08.170698 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:08.170699 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:08.171561 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:08.171563 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:08.172067 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:08.172068 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:08.172133 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:08.172137 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 16:52:08.172184 (+ 47us) client_negotiation.cc:240] Negotiation successful
0808 16:52:08.172222 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"mutex_wait_us":19,"thread_start_us":55,"threads_started":1}
I20260808 16:52:08.172523 11534 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7df36578fae3427c8f1e5b5927621a39" candidate_uuid: "74b028f8dcd8443493a06be381ff1317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25f72b14a05d4ec6861e7a7bbbac9ef5" is_pre_election: true
I20260808 16:52:08.172590 11534 raft_consensus.cc:2454] T 7df36578fae3427c8f1e5b5927621a39 P 25f72b14a05d4ec6861e7a7bbbac9ef5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74b028f8dcd8443493a06be381ff1317 in term 0.
I20260808 16:52:08.172715 11092 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7df36578fae3427c8f1e5b5927621a39" candidate_uuid: "74b028f8dcd8443493a06be381ff1317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "982252a2a0f04384a24931f2215efa1e" is_pre_election: true
I20260808 16:52:08.172760 11205 leader_election.cc:302] T 7df36578fae3427c8f1e5b5927621a39 P 74b028f8dcd8443493a06be381ff1317 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25f72b14a05d4ec6861e7a7bbbac9ef5, 74b028f8dcd8443493a06be381ff1317; no voters:
I20260808 16:52:08.172794 11092 raft_consensus.cc:2454] T 7df36578fae3427c8f1e5b5927621a39 P 982252a2a0f04384a24931f2215efa1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74b028f8dcd8443493a06be381ff1317 in term 0.
I20260808 16:52:08.172857 11654 raft_consensus.cc:2781] T 7df36578fae3427c8f1e5b5927621a39 P 74b028f8dcd8443493a06be381ff1317 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 16:52:08.172888 11654 raft_consensus.cc:482] T 7df36578fae3427c8f1e5b5927621a39 P 74b028f8dcd8443493a06be381ff1317 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 16:52:08.172904 11654 raft_consensus.cc:3037] T 7df36578fae3427c8f1e5b5927621a39 P 74b028f8dcd8443493a06be381ff1317 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:52:08.173525 11654 raft_consensus.cc:504] T 7df36578fae3427c8f1e5b5927621a39 P 74b028f8dcd8443493a06be381ff1317 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "982252a2a0f04384a24931f2215efa1e" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44087 } } peers { permanent_uuid: "74b028f8dcd8443493a06be381ff1317" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 44825 } } peers { permanent_uuid: "25f72b14a05d4ec6861e7a7bbbac9ef5" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 38397 } }
I20260808 16:52:08.173640 11654 leader_election.cc:288] T 7df36578fae3427c8f1e5b5927621a39 P 74b028f8dcd8443493a06be381ff1317 [CANDIDATE]: Term 1 election: Requested vote from peers 982252a2a0f04384a24931f2215efa1e (127.8.24.1:44087), 25f72b14a05d4ec6861e7a7bbbac9ef5 (127.8.24.3:38397)
I20260808 16:52:08.173766 11534 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7df36578fae3427c8f1e5b5927621a39" candidate_uuid: "74b028f8dcd8443493a06be381ff1317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25f72b14a05d4ec6861e7a7bbbac9ef5"
I20260808 16:52:08.173825 11534 raft_consensus.cc:3037] T 7df36578fae3427c8f1e5b5927621a39 P 25f72b14a05d4ec6861e7a7bbbac9ef5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:52:08.174414 11534 raft_consensus.cc:2454] T 7df36578fae3427c8f1e5b5927621a39 P 25f72b14a05d4ec6861e7a7bbbac9ef5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74b028f8dcd8443493a06be381ff1317 in term 1.
I20260808 16:52:08.174515 11092 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7df36578fae3427c8f1e5b5927621a39" candidate_uuid: "74b028f8dcd8443493a06be381ff1317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "982252a2a0f04384a24931f2215efa1e"
I20260808 16:52:08.174547 11205 leader_election.cc:302] T 7df36578fae3427c8f1e5b5927621a39 P 74b028f8dcd8443493a06be381ff1317 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25f72b14a05d4ec6861e7a7bbbac9ef5, 74b028f8dcd8443493a06be381ff1317; no voters:
I20260808 16:52:08.174593 11092 raft_consensus.cc:3037] T 7df36578fae3427c8f1e5b5927621a39 P 982252a2a0f04384a24931f2215efa1e [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:52:08.174638 11654 raft_consensus.cc:2781] T 7df36578fae3427c8f1e5b5927621a39 P 74b028f8dcd8443493a06be381ff1317 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:52:08.174765 11654 raft_consensus.cc:686] T 7df36578fae3427c8f1e5b5927621a39 P 74b028f8dcd8443493a06be381ff1317 [term 1 LEADER]: Becoming Leader. State: Replica: 74b028f8dcd8443493a06be381ff1317, State: Running, Role: LEADER
I20260808 16:52:08.174837 11654 consensus_queue.cc:237] T 7df36578fae3427c8f1e5b5927621a39 P 74b028f8dcd8443493a06be381ff1317 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "982252a2a0f04384a24931f2215efa1e" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44087 } } peers { permanent_uuid: "74b028f8dcd8443493a06be381ff1317" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 44825 } } peers { permanent_uuid: "25f72b14a05d4ec6861e7a7bbbac9ef5" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 38397 } }
I20260808 16:52:08.175304 11092 raft_consensus.cc:2454] T 7df36578fae3427c8f1e5b5927621a39 P 982252a2a0f04384a24931f2215efa1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74b028f8dcd8443493a06be381ff1317 in term 1.
I20260808 16:52:08.175725 10855 catalog_manager.cc:5495] T 7df36578fae3427c8f1e5b5927621a39 P 74b028f8dcd8443493a06be381ff1317 reported cstate change: term changed from 0 to 1, leader changed from <none> to 74b028f8dcd8443493a06be381ff1317 (127.8.24.2). New cstate: current_term: 1 leader_uuid: "74b028f8dcd8443493a06be381ff1317" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "982252a2a0f04384a24931f2215efa1e" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74b028f8dcd8443493a06be381ff1317" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 44825 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "25f72b14a05d4ec6861e7a7bbbac9ef5" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 38397 } health_report { overall_health: UNKNOWN } } }
I20260808 16:52:08.186504 11635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:08.184442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51696 (local address 127.8.24.2:44825)
0808 16:52:08.184476 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:08.184478 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:08.184485 (+ 7us) server_negotiation.cc:407] Connection header received
0808 16:52:08.184542 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:08.184542 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:08.184559 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:08.184621 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 16:52:08.184714 (+ 93us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:08.185142 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:08.185820 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:08.186009 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:08.186060 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:08.186198 (+ 138us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 16:52:08.186294 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 16:52:08.186374 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:08.186426 (+ 52us) server_negotiation.cc:299] Negotiation successful
0808 16:52:08.186444 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 16:52:08.190474 11638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:08.188582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:58105 (local address 127.8.24.1:44087)
0808 16:52:08.188628 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:08.188630 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:08.188728 (+ 98us) server_negotiation.cc:407] Connection header received
0808 16:52:08.188763 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:08.188763 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:08.188772 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:08.188811 (+ 39us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 16:52:08.188898 (+ 87us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:08.189309 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:08.189936 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:08.190103 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:08.190148 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:08.190282 (+ 134us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 16:52:08.190308 (+ 26us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 16:52:08.190377 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:08.190411 (+ 34us) server_negotiation.cc:299] Negotiation successful
0808 16:52:08.190428 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 16:52:08.190661 11733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:08.188520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:44087 (local address 127.8.24.2:58105)
0808 16:52:08.188668 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:08.188677 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:08.188727 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:08.188820 (+ 93us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:08.188820 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:08.188821 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 16:52:08.188867 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:08.188867 (+ 0us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:08.189354 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:08.189354 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:08.189880 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:08.189882 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:08.190171 (+ 289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:08.190171 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:08.190212 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:08.190224 (+ 12us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 16:52:08.190366 (+ 142us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 16:52:08.190368 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 16:52:08.190407 (+ 39us) client_negotiation.cc:240] Negotiation successful
0808 16:52:08.190440 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":82,"threads_started":1}
I20260808 16:52:08.192313 11534 raft_consensus.cc:1261] T 7df36578fae3427c8f1e5b5927621a39 P 25f72b14a05d4ec6861e7a7bbbac9ef5 [term 1 FOLLOWER]: Refusing update from remote peer 74b028f8dcd8443493a06be381ff1317: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:52:08.192318 11092 raft_consensus.cc:1261] T 7df36578fae3427c8f1e5b5927621a39 P 982252a2a0f04384a24931f2215efa1e [term 1 FOLLOWER]: Refusing update from remote peer 74b028f8dcd8443493a06be381ff1317: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:52:08.192544 11726 consensus_queue.cc:1035] T 7df36578fae3427c8f1e5b5927621a39 P 74b028f8dcd8443493a06be381ff1317 [LEADER]: Connected to new peer: Peer: permanent_uuid: "982252a2a0f04384a24931f2215efa1e" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:52:08.192663 11654 consensus_queue.cc:1035] T 7df36578fae3427c8f1e5b5927621a39 P 74b028f8dcd8443493a06be381ff1317 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25f72b14a05d4ec6861e7a7bbbac9ef5" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 38397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:52:08.199232 11635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:08.197061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:52947 (local address 127.8.24.2:44825)
0808 16:52:08.197112 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:08.197114 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:08.197122 (+ 8us) server_negotiation.cc:407] Connection header received
0808 16:52:08.197159 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:08.197160 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:08.197168 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:08.197206 (+ 38us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 16:52:08.197321 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:08.197762 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:08.198497 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:08.198754 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:08.198812 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:08.198944 (+ 132us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 16:52:08.198970 (+ 26us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 16:52:08.199014 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:08.199174 (+ 160us) server_negotiation.cc:299] Negotiation successful
0808 16:52:08.199192 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 16:52:08.199263 11705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:08.197033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.2:44825 (local address 127.8.24.1:52947)
0808 16:52:08.197071 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:08.197081 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:08.197116 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:08.197222 (+ 106us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:08.197222 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:08.197224 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 16:52:08.197263 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:08.197265 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:08.197822 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:08.197823 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:08.198421 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:08.198423 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:08.198825 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:08.198826 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:08.198886 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:08.198891 (+ 5us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 16:52:08.199117 (+ 226us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 16:52:08.199120 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 16:52:08.199164 (+ 44us) client_negotiation.cc:240] Negotiation successful
0808 16:52:08.199206 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18}
W20260808 16:52:08.199548 11683 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260808 16:52:09.717113 10856 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59230:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 16:52:09.717213 10856 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59230:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 16:52:09.717983 10856 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5627ffa85f6546a1a64643aa34ab1e6f: Sending DeleteTablet for 3 replicas of tablet 7df36578fae3427c8f1e5b5927621a39
I20260808 16:52:09.718211 11060 tablet_service.cc:1514] Processing DeleteTablet for tablet 7df36578fae3427c8f1e5b5927621a39 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:52:09 UTC) from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:43148
I20260808 16:52:09.718227 11501 tablet_service.cc:1514] Processing DeleteTablet for tablet 7df36578fae3427c8f1e5b5927621a39 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:52:09 UTC) from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:50978
I20260808 16:52:09.718425 11781 tablet_replica.cc:330] T 7df36578fae3427c8f1e5b5927621a39 P 982252a2a0f04384a24931f2215efa1e: stopping tablet replica
I20260808 16:52:09.718459 11784 tablet_replica.cc:330] T 7df36578fae3427c8f1e5b5927621a39 P 25f72b14a05d4ec6861e7a7bbbac9ef5: stopping tablet replica
I20260808 16:52:09.718508 11781 raft_consensus.cc:2229] T 7df36578fae3427c8f1e5b5927621a39 P 982252a2a0f04384a24931f2215efa1e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 16:52:09.718528 11784 raft_consensus.cc:2229] T 7df36578fae3427c8f1e5b5927621a39 P 25f72b14a05d4ec6861e7a7bbbac9ef5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 16:52:09.718555 11781 raft_consensus.cc:2258] T 7df36578fae3427c8f1e5b5927621a39 P 982252a2a0f04384a24931f2215efa1e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:52:09.718564 11784 raft_consensus.cc:2258] T 7df36578fae3427c8f1e5b5927621a39 P 25f72b14a05d4ec6861e7a7bbbac9ef5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:52:09.718779 11784 ts_tablet_manager.cc:1905] T 7df36578fae3427c8f1e5b5927621a39 P 25f72b14a05d4ec6861e7a7bbbac9ef5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:52:09.718924 11781 ts_tablet_manager.cc:1905] T 7df36578fae3427c8f1e5b5927621a39 P 982252a2a0f04384a24931f2215efa1e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:52:09.720166 11278 tablet_service.cc:1514] Processing DeleteTablet for tablet 7df36578fae3427c8f1e5b5927621a39 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:52:09 UTC) from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:51682
I20260808 16:52:09.720347 11784 ts_tablet_manager.cc:1918] T 7df36578fae3427c8f1e5b5927621a39 P 25f72b14a05d4ec6861e7a7bbbac9ef5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 16:52:09.720399 11784 log.cc:1196] T 7df36578fae3427c8f1e5b5927621a39 P 25f72b14a05d4ec6861e7a7bbbac9ef5: Deleting WAL directory at /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/minicluster-data/ts-2/wal/wals/7df36578fae3427c8f1e5b5927621a39
I20260808 16:52:09.720618 11784 ts_tablet_manager.cc:1939] T 7df36578fae3427c8f1e5b5927621a39 P 25f72b14a05d4ec6861e7a7bbbac9ef5: Deleting consensus metadata
I20260808 16:52:09.720660 11781 ts_tablet_manager.cc:1918] T 7df36578fae3427c8f1e5b5927621a39 P 982252a2a0f04384a24931f2215efa1e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 16:52:09.720712 11781 log.cc:1196] T 7df36578fae3427c8f1e5b5927621a39 P 982252a2a0f04384a24931f2215efa1e: Deleting WAL directory at /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/minicluster-data/ts-0/wal/wals/7df36578fae3427c8f1e5b5927621a39
I20260808 16:52:09.720909 11781 ts_tablet_manager.cc:1939] T 7df36578fae3427c8f1e5b5927621a39 P 982252a2a0f04384a24931f2215efa1e: Deleting consensus metadata
I20260808 16:52:09.720890 10841 catalog_manager.cc:4841] TS 25f72b14a05d4ec6861e7a7bbbac9ef5 (127.8.24.3:38397): tablet 7df36578fae3427c8f1e5b5927621a39 (table test-table [id=8d8eefa06a764f0099927da857db17a8]) successfully deleted
I20260808 16:52:09.721225 10840 catalog_manager.cc:4841] TS 982252a2a0f04384a24931f2215efa1e (127.8.24.1:44087): tablet 7df36578fae3427c8f1e5b5927621a39 (table test-table [id=8d8eefa06a764f0099927da857db17a8]) successfully deleted
I20260808 16:52:09.721735 11788 tablet_replica.cc:330] T 7df36578fae3427c8f1e5b5927621a39 P 74b028f8dcd8443493a06be381ff1317: stopping tablet replica
I20260808 16:52:09.721786 11788 raft_consensus.cc:2229] T 7df36578fae3427c8f1e5b5927621a39 P 74b028f8dcd8443493a06be381ff1317 [term 1 LEADER]: Raft consensus shutting down.
I20260808 16:52:09.721846 11788 raft_consensus.cc:2258] T 7df36578fae3427c8f1e5b5927621a39 P 74b028f8dcd8443493a06be381ff1317 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:52:09.722031 11788 ts_tablet_manager.cc:1905] T 7df36578fae3427c8f1e5b5927621a39 P 74b028f8dcd8443493a06be381ff1317: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:52:09.723177 11788 ts_tablet_manager.cc:1918] T 7df36578fae3427c8f1e5b5927621a39 P 74b028f8dcd8443493a06be381ff1317: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 16:52:09.723228 11788 log.cc:1196] T 7df36578fae3427c8f1e5b5927621a39 P 74b028f8dcd8443493a06be381ff1317: Deleting WAL directory at /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/minicluster-data/ts-1/wal/wals/7df36578fae3427c8f1e5b5927621a39
I20260808 16:52:09.723376 11788 ts_tablet_manager.cc:1939] T 7df36578fae3427c8f1e5b5927621a39 P 74b028f8dcd8443493a06be381ff1317: Deleting consensus metadata
I20260808 16:52:09.723632 10837 catalog_manager.cc:4841] TS 74b028f8dcd8443493a06be381ff1317 (127.8.24.2:44825): tablet 7df36578fae3427c8f1e5b5927621a39 (table test-table [id=8d8eefa06a764f0099927da857db17a8]) successfully deleted
Aug 08 16:52:09 dist-test-slave-6zbq krb5kdc[10795](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207929, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 16:52:09 dist-test-slave-6zbq krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207929, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260808 16:52:09.740610 11797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:09.734502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59240 (local address 127.8.24.62:41799)
0808 16:52:09.734635 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:09.734637 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:09.734645 (+ 8us) server_negotiation.cc:407] Connection header received
0808 16:52:09.734693 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:09.734693 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:09.734702 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:09.734755 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:09.735211 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:09.735701 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:09.736344 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:09.736520 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:09.736560 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:09.738489 (+ 1929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:09.738491 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:09.738491 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:09.738498 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:09.739611 (+ 1113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:09.739928 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:09.739929 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:09.739930 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:09.739985 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:09.740145 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:09.740146 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:09.740147 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:09.740250 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:09.740345 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:09.740441 (+ 96us) server_negotiation.cc:299] Negotiation successful
0808 16:52:09.740517 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":77,"threads_started":1}
I20260808 16:52:09.741778 10856 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59240:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 16:52:09.741904 10856 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 16:52:09.743012 11060 tablet_service.cc:1467] Processing CreateTablet for tablet d3d0e36b9e224db0b65c02a2770f3f9b (DEFAULT_TABLE table=test-table [id=9b32f11c2d064f5e996be7ec5c16200b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:52:09.743050 11278 tablet_service.cc:1467] Processing CreateTablet for tablet d3d0e36b9e224db0b65c02a2770f3f9b (DEFAULT_TABLE table=test-table [id=9b32f11c2d064f5e996be7ec5c16200b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:52:09.743072 11501 tablet_service.cc:1467] Processing CreateTablet for tablet d3d0e36b9e224db0b65c02a2770f3f9b (DEFAULT_TABLE table=test-table [id=9b32f11c2d064f5e996be7ec5c16200b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:52:09.743121 11060 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3d0e36b9e224db0b65c02a2770f3f9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:52:09.743148 11278 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3d0e36b9e224db0b65c02a2770f3f9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:52:09.743160 11501 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3d0e36b9e224db0b65c02a2770f3f9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:52:09.744050 11798 tablet_bootstrap.cc:492] T d3d0e36b9e224db0b65c02a2770f3f9b P 74b028f8dcd8443493a06be381ff1317: Bootstrap starting.
I20260808 16:52:09.744375 11798 tablet_bootstrap.cc:654] T d3d0e36b9e224db0b65c02a2770f3f9b P 74b028f8dcd8443493a06be381ff1317: Neither blocks nor log segments found. Creating new log.
I20260808 16:52:09.744551 11799 tablet_bootstrap.cc:492] T d3d0e36b9e224db0b65c02a2770f3f9b P 982252a2a0f04384a24931f2215efa1e: Bootstrap starting.
I20260808 16:52:09.744863 11799 tablet_bootstrap.cc:654] T d3d0e36b9e224db0b65c02a2770f3f9b P 982252a2a0f04384a24931f2215efa1e: Neither blocks nor log segments found. Creating new log.
I20260808 16:52:09.744928 11798 tablet_bootstrap.cc:492] T d3d0e36b9e224db0b65c02a2770f3f9b P 74b028f8dcd8443493a06be381ff1317: No bootstrap required, opened a new log
I20260808 16:52:09.744978 11798 ts_tablet_manager.cc:1397] T d3d0e36b9e224db0b65c02a2770f3f9b P 74b028f8dcd8443493a06be381ff1317: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 16:52:09.745081 11798 raft_consensus.cc:348] T d3d0e36b9e224db0b65c02a2770f3f9b P 74b028f8dcd8443493a06be381ff1317 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "982252a2a0f04384a24931f2215efa1e" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44087 } } peers { permanent_uuid: "74b028f8dcd8443493a06be381ff1317" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 44825 } } peers { permanent_uuid: "25f72b14a05d4ec6861e7a7bbbac9ef5" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 38397 } }
I20260808 16:52:09.745133 11800 tablet_bootstrap.cc:492] T d3d0e36b9e224db0b65c02a2770f3f9b P 25f72b14a05d4ec6861e7a7bbbac9ef5: Bootstrap starting.
I20260808 16:52:09.745151 11798 raft_consensus.cc:374] T d3d0e36b9e224db0b65c02a2770f3f9b P 74b028f8dcd8443493a06be381ff1317 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:52:09.745172 11798 raft_consensus.cc:729] T d3d0e36b9e224db0b65c02a2770f3f9b P 74b028f8dcd8443493a06be381ff1317 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74b028f8dcd8443493a06be381ff1317, State: Initialized, Role: FOLLOWER
I20260808 16:52:09.745231 11798 consensus_queue.cc:260] T d3d0e36b9e224db0b65c02a2770f3f9b P 74b028f8dcd8443493a06be381ff1317 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "982252a2a0f04384a24931f2215efa1e" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44087 } } peers { permanent_uuid: "74b028f8dcd8443493a06be381ff1317" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 44825 } } peers { permanent_uuid: "25f72b14a05d4ec6861e7a7bbbac9ef5" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 38397 } }
I20260808 16:52:09.745373 11798 ts_tablet_manager.cc:1428] T d3d0e36b9e224db0b65c02a2770f3f9b P 74b028f8dcd8443493a06be381ff1317: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:09.745416 11799 tablet_bootstrap.cc:492] T d3d0e36b9e224db0b65c02a2770f3f9b P 982252a2a0f04384a24931f2215efa1e: No bootstrap required, opened a new log
I20260808 16:52:09.745484 11799 ts_tablet_manager.cc:1397] T d3d0e36b9e224db0b65c02a2770f3f9b P 982252a2a0f04384a24931f2215efa1e: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 16:52:09.745496 11800 tablet_bootstrap.cc:654] T d3d0e36b9e224db0b65c02a2770f3f9b P 25f72b14a05d4ec6861e7a7bbbac9ef5: Neither blocks nor log segments found. Creating new log.
I20260808 16:52:09.745607 11799 raft_consensus.cc:348] T d3d0e36b9e224db0b65c02a2770f3f9b P 982252a2a0f04384a24931f2215efa1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "982252a2a0f04384a24931f2215efa1e" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44087 } } peers { permanent_uuid: "74b028f8dcd8443493a06be381ff1317" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 44825 } } peers { permanent_uuid: "25f72b14a05d4ec6861e7a7bbbac9ef5" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 38397 } }
I20260808 16:52:09.745676 11799 raft_consensus.cc:374] T d3d0e36b9e224db0b65c02a2770f3f9b P 982252a2a0f04384a24931f2215efa1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:52:09.745694 11799 raft_consensus.cc:729] T d3d0e36b9e224db0b65c02a2770f3f9b P 982252a2a0f04384a24931f2215efa1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 982252a2a0f04384a24931f2215efa1e, State: Initialized, Role: FOLLOWER
I20260808 16:52:09.745762 11799 consensus_queue.cc:260] T d3d0e36b9e224db0b65c02a2770f3f9b P 982252a2a0f04384a24931f2215efa1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "982252a2a0f04384a24931f2215efa1e" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44087 } } peers { permanent_uuid: "74b028f8dcd8443493a06be381ff1317" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 44825 } } peers { permanent_uuid: "25f72b14a05d4ec6861e7a7bbbac9ef5" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 38397 } }
I20260808 16:52:09.745985 11799 ts_tablet_manager.cc:1428] T d3d0e36b9e224db0b65c02a2770f3f9b P 982252a2a0f04384a24931f2215efa1e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:09.746107 11800 tablet_bootstrap.cc:492] T d3d0e36b9e224db0b65c02a2770f3f9b P 25f72b14a05d4ec6861e7a7bbbac9ef5: No bootstrap required, opened a new log
I20260808 16:52:09.746186 11800 ts_tablet_manager.cc:1397] T d3d0e36b9e224db0b65c02a2770f3f9b P 25f72b14a05d4ec6861e7a7bbbac9ef5: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 16:52:09.746332 11800 raft_consensus.cc:348] T d3d0e36b9e224db0b65c02a2770f3f9b P 25f72b14a05d4ec6861e7a7bbbac9ef5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "982252a2a0f04384a24931f2215efa1e" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44087 } } peers { permanent_uuid: "74b028f8dcd8443493a06be381ff1317" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 44825 } } peers { permanent_uuid: "25f72b14a05d4ec6861e7a7bbbac9ef5" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 38397 } }
I20260808 16:52:09.746467 11800 raft_consensus.cc:374] T d3d0e36b9e224db0b65c02a2770f3f9b P 25f72b14a05d4ec6861e7a7bbbac9ef5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:52:09.746503 11800 raft_consensus.cc:729] T d3d0e36b9e224db0b65c02a2770f3f9b P 25f72b14a05d4ec6861e7a7bbbac9ef5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25f72b14a05d4ec6861e7a7bbbac9ef5, State: Initialized, Role: FOLLOWER
I20260808 16:52:09.746557 11800 consensus_queue.cc:260] T d3d0e36b9e224db0b65c02a2770f3f9b P 25f72b14a05d4ec6861e7a7bbbac9ef5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "982252a2a0f04384a24931f2215efa1e" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44087 } } peers { permanent_uuid: "74b028f8dcd8443493a06be381ff1317" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 44825 } } peers { permanent_uuid: "25f72b14a05d4ec6861e7a7bbbac9ef5" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 38397 } }
I20260808 16:52:09.746727 11800 ts_tablet_manager.cc:1428] T d3d0e36b9e224db0b65c02a2770f3f9b P 25f72b14a05d4ec6861e7a7bbbac9ef5: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:09.883361 11751 raft_consensus.cc:482] T d3d0e36b9e224db0b65c02a2770f3f9b P 982252a2a0f04384a24931f2215efa1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 16:52:09.883451 11751 raft_consensus.cc:504] T d3d0e36b9e224db0b65c02a2770f3f9b P 982252a2a0f04384a24931f2215efa1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "982252a2a0f04384a24931f2215efa1e" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44087 } } peers { permanent_uuid: "74b028f8dcd8443493a06be381ff1317" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 44825 } } peers { permanent_uuid: "25f72b14a05d4ec6861e7a7bbbac9ef5" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 38397 } }
I20260808 16:52:09.883616 11751 leader_election.cc:288] T d3d0e36b9e224db0b65c02a2770f3f9b P 982252a2a0f04384a24931f2215efa1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 74b028f8dcd8443493a06be381ff1317 (127.8.24.2:44825), 25f72b14a05d4ec6861e7a7bbbac9ef5 (127.8.24.3:38397)
I20260808 16:52:09.883819 11309 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3d0e36b9e224db0b65c02a2770f3f9b" candidate_uuid: "982252a2a0f04384a24931f2215efa1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74b028f8dcd8443493a06be381ff1317" is_pre_election: true
I20260808 16:52:09.883826 11534 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3d0e36b9e224db0b65c02a2770f3f9b" candidate_uuid: "982252a2a0f04384a24931f2215efa1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25f72b14a05d4ec6861e7a7bbbac9ef5" is_pre_election: true
I20260808 16:52:09.883906 11309 raft_consensus.cc:2454] T d3d0e36b9e224db0b65c02a2770f3f9b P 74b028f8dcd8443493a06be381ff1317 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 982252a2a0f04384a24931f2215efa1e in term 0.
I20260808 16:52:09.883927 11534 raft_consensus.cc:2454] T d3d0e36b9e224db0b65c02a2770f3f9b P 25f72b14a05d4ec6861e7a7bbbac9ef5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 982252a2a0f04384a24931f2215efa1e in term 0.
I20260808 16:52:09.884056 10992 leader_election.cc:302] T d3d0e36b9e224db0b65c02a2770f3f9b P 982252a2a0f04384a24931f2215efa1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74b028f8dcd8443493a06be381ff1317, 982252a2a0f04384a24931f2215efa1e; no voters:
I20260808 16:52:09.884171 11751 raft_consensus.cc:2781] T d3d0e36b9e224db0b65c02a2770f3f9b P 982252a2a0f04384a24931f2215efa1e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 16:52:09.884222 11751 raft_consensus.cc:482] T d3d0e36b9e224db0b65c02a2770f3f9b P 982252a2a0f04384a24931f2215efa1e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 16:52:09.884241 11751 raft_consensus.cc:3037] T d3d0e36b9e224db0b65c02a2770f3f9b P 982252a2a0f04384a24931f2215efa1e [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:52:09.884946 11751 raft_consensus.cc:504] T d3d0e36b9e224db0b65c02a2770f3f9b P 982252a2a0f04384a24931f2215efa1e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "982252a2a0f04384a24931f2215efa1e" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44087 } } peers { permanent_uuid: "74b028f8dcd8443493a06be381ff1317" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 44825 } } peers { permanent_uuid: "25f72b14a05d4ec6861e7a7bbbac9ef5" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 38397 } }
I20260808 16:52:09.885066 11751 leader_election.cc:288] T d3d0e36b9e224db0b65c02a2770f3f9b P 982252a2a0f04384a24931f2215efa1e [CANDIDATE]: Term 1 election: Requested vote from peers 74b028f8dcd8443493a06be381ff1317 (127.8.24.2:44825), 25f72b14a05d4ec6861e7a7bbbac9ef5 (127.8.24.3:38397)
I20260808 16:52:09.885214 11534 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3d0e36b9e224db0b65c02a2770f3f9b" candidate_uuid: "982252a2a0f04384a24931f2215efa1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25f72b14a05d4ec6861e7a7bbbac9ef5"
I20260808 16:52:09.885215 11309 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3d0e36b9e224db0b65c02a2770f3f9b" candidate_uuid: "982252a2a0f04384a24931f2215efa1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74b028f8dcd8443493a06be381ff1317"
I20260808 16:52:09.885293 11309 raft_consensus.cc:3037] T d3d0e36b9e224db0b65c02a2770f3f9b P 74b028f8dcd8443493a06be381ff1317 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:52:09.885293 11534 raft_consensus.cc:3037] T d3d0e36b9e224db0b65c02a2770f3f9b P 25f72b14a05d4ec6861e7a7bbbac9ef5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:52:09.886021 11534 raft_consensus.cc:2454] T d3d0e36b9e224db0b65c02a2770f3f9b P 25f72b14a05d4ec6861e7a7bbbac9ef5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 982252a2a0f04384a24931f2215efa1e in term 1.
I20260808 16:52:09.886022 11309 raft_consensus.cc:2454] T d3d0e36b9e224db0b65c02a2770f3f9b P 74b028f8dcd8443493a06be381ff1317 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 982252a2a0f04384a24931f2215efa1e in term 1.
I20260808 16:52:09.886188 10995 leader_election.cc:302] T d3d0e36b9e224db0b65c02a2770f3f9b P 982252a2a0f04384a24931f2215efa1e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25f72b14a05d4ec6861e7a7bbbac9ef5, 982252a2a0f04384a24931f2215efa1e; no voters:
I20260808 16:52:09.886289 11751 raft_consensus.cc:2781] T d3d0e36b9e224db0b65c02a2770f3f9b P 982252a2a0f04384a24931f2215efa1e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:52:09.886359 11751 raft_consensus.cc:686] T d3d0e36b9e224db0b65c02a2770f3f9b P 982252a2a0f04384a24931f2215efa1e [term 1 LEADER]: Becoming Leader. State: Replica: 982252a2a0f04384a24931f2215efa1e, State: Running, Role: LEADER
I20260808 16:52:09.886407 11751 consensus_queue.cc:237] T d3d0e36b9e224db0b65c02a2770f3f9b P 982252a2a0f04384a24931f2215efa1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "982252a2a0f04384a24931f2215efa1e" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44087 } } peers { permanent_uuid: "74b028f8dcd8443493a06be381ff1317" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 44825 } } peers { permanent_uuid: "25f72b14a05d4ec6861e7a7bbbac9ef5" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 38397 } }
I20260808 16:52:09.887015 10856 catalog_manager.cc:5495] T d3d0e36b9e224db0b65c02a2770f3f9b P 982252a2a0f04384a24931f2215efa1e reported cstate change: term changed from 0 to 1, leader changed from <none> to 982252a2a0f04384a24931f2215efa1e (127.8.24.1). New cstate: current_term: 1 leader_uuid: "982252a2a0f04384a24931f2215efa1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "982252a2a0f04384a24931f2215efa1e" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44087 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74b028f8dcd8443493a06be381ff1317" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 44825 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25f72b14a05d4ec6861e7a7bbbac9ef5" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 38397 } health_report { overall_health: UNKNOWN } } }
I20260808 16:52:09.928153 11814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:09.926097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43164 (local address 127.8.24.1:44087)
0808 16:52:09.926255 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:09.926257 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:09.926271 (+ 14us) server_negotiation.cc:407] Connection header received
0808 16:52:09.926290 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:09.926290 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:09.926300 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:09.926377 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 16:52:09.926490 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:09.926948 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:09.927636 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:09.927826 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:09.927877 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:09.927964 (+ 87us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 16:52:09.927988 (+ 24us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 16:52:09.928018 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:09.928062 (+ 44us) server_negotiation.cc:299] Negotiation successful
0808 16:52:09.928076 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":87,"threads_started":1}
I20260808 16:52:09.931897 11814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:09.930148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:39087 (local address 127.8.24.1:44087)
0808 16:52:09.930179 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:09.930181 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:09.930208 (+ 27us) server_negotiation.cc:407] Connection header received
0808 16:52:09.930255 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:09.930256 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:09.930272 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:09.930307 (+ 35us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 16:52:09.930428 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:09.930837 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:09.931456 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:09.931617 (+ 161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:09.931655 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:09.931739 (+ 84us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 16:52:09.931759 (+ 20us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 16:52:09.931790 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:09.931822 (+ 32us) server_negotiation.cc:299] Negotiation successful
0808 16:52:09.931834 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260808 16:52:09.931946 11819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:09.930032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:44087 (local address 127.8.24.1:39087)
0808 16:52:09.930161 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:09.930170 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:09.930220 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:09.930316 (+ 96us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:09.930317 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:09.930318 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 16:52:09.930380 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:09.930381 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:09.930891 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:09.930892 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:09.931406 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:09.931407 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:09.931661 (+ 254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:09.931661 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:09.931694 (+ 33us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:09.931697 (+ 3us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 16:52:09.931791 (+ 94us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 16:52:09.931792 (+ 1us) client_negotiation.cc:769] Sending connection context
0808 16:52:09.931819 (+ 27us) client_negotiation.cc:240] Negotiation successful
0808 16:52:09.931848 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":71,"threads_started":1}
I20260808 16:52:09.933763 11534 raft_consensus.cc:1261] T d3d0e36b9e224db0b65c02a2770f3f9b P 25f72b14a05d4ec6861e7a7bbbac9ef5 [term 1 FOLLOWER]: Refusing update from remote peer 982252a2a0f04384a24931f2215efa1e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:52:09.933763 11309 raft_consensus.cc:1261] T d3d0e36b9e224db0b65c02a2770f3f9b P 74b028f8dcd8443493a06be381ff1317 [term 1 FOLLOWER]: Refusing update from remote peer 982252a2a0f04384a24931f2215efa1e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:52:09.934087 11759 consensus_queue.cc:1035] T d3d0e36b9e224db0b65c02a2770f3f9b P 982252a2a0f04384a24931f2215efa1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "25f72b14a05d4ec6861e7a7bbbac9ef5" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 38397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:52:09.934196 11742 consensus_queue.cc:1035] T d3d0e36b9e224db0b65c02a2770f3f9b P 982252a2a0f04384a24931f2215efa1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "74b028f8dcd8443493a06be381ff1317" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 44825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:52:09.943919 10856 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59240:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 16:52:09.943996 10856 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59240:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 16:52:09.944558 10856 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5627ffa85f6546a1a64643aa34ab1e6f: Sending DeleteTablet for 3 replicas of tablet d3d0e36b9e224db0b65c02a2770f3f9b
I20260808 16:52:09.944784 11501 tablet_service.cc:1514] Processing DeleteTablet for tablet d3d0e36b9e224db0b65c02a2770f3f9b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:52:09 UTC) from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:50978
I20260808 16:52:09.944876 11278 tablet_service.cc:1514] Processing DeleteTablet for tablet d3d0e36b9e224db0b65c02a2770f3f9b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:52:09 UTC) from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:51682
I20260808 16:52:09.944885 11784 tablet_replica.cc:330] T d3d0e36b9e224db0b65c02a2770f3f9b P 25f72b14a05d4ec6861e7a7bbbac9ef5: stopping tablet replica
I20260808 16:52:09.944953 11784 raft_consensus.cc:2229] T d3d0e36b9e224db0b65c02a2770f3f9b P 25f72b14a05d4ec6861e7a7bbbac9ef5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 16:52:09.944960 11788 tablet_replica.cc:330] T d3d0e36b9e224db0b65c02a2770f3f9b P 74b028f8dcd8443493a06be381ff1317: stopping tablet replica
I20260808 16:52:09.944994 11784 raft_consensus.cc:2258] T d3d0e36b9e224db0b65c02a2770f3f9b P 25f72b14a05d4ec6861e7a7bbbac9ef5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:52:09.945005 11788 raft_consensus.cc:2229] T d3d0e36b9e224db0b65c02a2770f3f9b P 74b028f8dcd8443493a06be381ff1317 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 16:52:09.945036 11788 raft_consensus.cc:2258] T d3d0e36b9e224db0b65c02a2770f3f9b P 74b028f8dcd8443493a06be381ff1317 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:52:09.945158 11059 tablet_service.cc:1514] Processing DeleteTablet for tablet d3d0e36b9e224db0b65c02a2770f3f9b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:52:09 UTC) from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:43148
I20260808 16:52:09.945245 11781 tablet_replica.cc:330] T d3d0e36b9e224db0b65c02a2770f3f9b P 982252a2a0f04384a24931f2215efa1e: stopping tablet replica
I20260808 16:52:09.945231 10856 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59240:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 16:52:09.945314 11781 raft_consensus.cc:2229] T d3d0e36b9e224db0b65c02a2770f3f9b P 982252a2a0f04384a24931f2215efa1e [term 1 LEADER]: Raft consensus shutting down.
W20260808 16:52:09.945365 10856 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 16:52:09.945390 11781 raft_consensus.cc:2258] T d3d0e36b9e224db0b65c02a2770f3f9b P 982252a2a0f04384a24931f2215efa1e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:52:09.945760 11784 ts_tablet_manager.cc:1905] T d3d0e36b9e224db0b65c02a2770f3f9b P 25f72b14a05d4ec6861e7a7bbbac9ef5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:52:09.945761 11788 ts_tablet_manager.cc:1905] T d3d0e36b9e224db0b65c02a2770f3f9b P 74b028f8dcd8443493a06be381ff1317: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:52:09.945828 11781 ts_tablet_manager.cc:1905] T d3d0e36b9e224db0b65c02a2770f3f9b P 982252a2a0f04384a24931f2215efa1e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:52:09.947395 11784 ts_tablet_manager.cc:1918] T d3d0e36b9e224db0b65c02a2770f3f9b P 25f72b14a05d4ec6861e7a7bbbac9ef5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 16:52:09.947395 11788 ts_tablet_manager.cc:1918] T d3d0e36b9e224db0b65c02a2770f3f9b P 74b028f8dcd8443493a06be381ff1317: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 16:52:09.947450 11784 log.cc:1196] T d3d0e36b9e224db0b65c02a2770f3f9b P 25f72b14a05d4ec6861e7a7bbbac9ef5: Deleting WAL directory at /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/minicluster-data/ts-2/wal/wals/d3d0e36b9e224db0b65c02a2770f3f9b
I20260808 16:52:09.947450 11788 log.cc:1196] T d3d0e36b9e224db0b65c02a2770f3f9b P 74b028f8dcd8443493a06be381ff1317: Deleting WAL directory at /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/minicluster-data/ts-1/wal/wals/d3d0e36b9e224db0b65c02a2770f3f9b
I20260808 16:52:09.947649 11788 ts_tablet_manager.cc:1939] T d3d0e36b9e224db0b65c02a2770f3f9b P 74b028f8dcd8443493a06be381ff1317: Deleting consensus metadata
I20260808 16:52:09.947677 11784 ts_tablet_manager.cc:1939] T d3d0e36b9e224db0b65c02a2770f3f9b P 25f72b14a05d4ec6861e7a7bbbac9ef5: Deleting consensus metadata
I20260808 16:52:09.947957 10837 catalog_manager.cc:4841] TS 74b028f8dcd8443493a06be381ff1317 (127.8.24.2:44825): tablet d3d0e36b9e224db0b65c02a2770f3f9b (table test-table [id=9b32f11c2d064f5e996be7ec5c16200b]) successfully deleted
I20260808 16:52:09.948009 10841 catalog_manager.cc:4841] TS 25f72b14a05d4ec6861e7a7bbbac9ef5 (127.8.24.3:38397): tablet d3d0e36b9e224db0b65c02a2770f3f9b (table test-table [id=9b32f11c2d064f5e996be7ec5c16200b]) successfully deleted
I20260808 16:52:09.948024 11501 tablet_service.cc:1467] Processing CreateTablet for tablet dea5cf8d544d4238bc4e90aa202fab00 (DEFAULT_TABLE table=test-table [id=3e1281bea5f84cd09dff937c9c631957]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:52:09.948125 11501 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dea5cf8d544d4238bc4e90aa202fab00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:52:09.948124 11781 ts_tablet_manager.cc:1918] T d3d0e36b9e224db0b65c02a2770f3f9b P 982252a2a0f04384a24931f2215efa1e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 16:52:09.948164 11781 log.cc:1196] T d3d0e36b9e224db0b65c02a2770f3f9b P 982252a2a0f04384a24931f2215efa1e: Deleting WAL directory at /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/minicluster-data/ts-0/wal/wals/d3d0e36b9e224db0b65c02a2770f3f9b
I20260808 16:52:09.948150 11278 tablet_service.cc:1467] Processing CreateTablet for tablet dea5cf8d544d4238bc4e90aa202fab00 (DEFAULT_TABLE table=test-table [id=3e1281bea5f84cd09dff937c9c631957]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:52:09.948239 11278 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dea5cf8d544d4238bc4e90aa202fab00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:52:09.948354 11781 ts_tablet_manager.cc:1939] T d3d0e36b9e224db0b65c02a2770f3f9b P 982252a2a0f04384a24931f2215efa1e: Deleting consensus metadata
I20260808 16:52:09.948606 10840 catalog_manager.cc:4841] TS 982252a2a0f04384a24931f2215efa1e (127.8.24.1:44087): tablet d3d0e36b9e224db0b65c02a2770f3f9b (table test-table [id=9b32f11c2d064f5e996be7ec5c16200b]) successfully deleted
I20260808 16:52:09.949085 11800 tablet_bootstrap.cc:492] T dea5cf8d544d4238bc4e90aa202fab00 P 25f72b14a05d4ec6861e7a7bbbac9ef5: Bootstrap starting.
I20260808 16:52:09.949316 11798 tablet_bootstrap.cc:492] T dea5cf8d544d4238bc4e90aa202fab00 P 74b028f8dcd8443493a06be381ff1317: Bootstrap starting.
I20260808 16:52:09.949388 11800 tablet_bootstrap.cc:654] T dea5cf8d544d4238bc4e90aa202fab00 P 25f72b14a05d4ec6861e7a7bbbac9ef5: Neither blocks nor log segments found. Creating new log.
I20260808 16:52:09.949620 11798 tablet_bootstrap.cc:654] T dea5cf8d544d4238bc4e90aa202fab00 P 74b028f8dcd8443493a06be381ff1317: Neither blocks nor log segments found. Creating new log.
I20260808 16:52:09.949854 11800 tablet_bootstrap.cc:492] T dea5cf8d544d4238bc4e90aa202fab00 P 25f72b14a05d4ec6861e7a7bbbac9ef5: No bootstrap required, opened a new log
I20260808 16:52:09.949895 11800 ts_tablet_manager.cc:1397] T dea5cf8d544d4238bc4e90aa202fab00 P 25f72b14a05d4ec6861e7a7bbbac9ef5: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260808 16:52:09.949986 11800 raft_consensus.cc:348] T dea5cf8d544d4238bc4e90aa202fab00 P 25f72b14a05d4ec6861e7a7bbbac9ef5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "982252a2a0f04384a24931f2215efa1e" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44087 } } peers { permanent_uuid: "74b028f8dcd8443493a06be381ff1317" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 44825 } } peers { permanent_uuid: "25f72b14a05d4ec6861e7a7bbbac9ef5" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 38397 } }
I20260808 16:52:09.950037 11800 raft_consensus.cc:374] T dea5cf8d544d4238bc4e90aa202fab00 P 25f72b14a05d4ec6861e7a7bbbac9ef5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:52:09.950049 11800 raft_consensus.cc:729] T dea5cf8d544d4238bc4e90aa202fab00 P 25f72b14a05d4ec6861e7a7bbbac9ef5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25f72b14a05d4ec6861e7a7bbbac9ef5, State: Initialized, Role: FOLLOWER
I20260808 16:52:09.950044 11059 tablet_service.cc:1467] Processing CreateTablet for tablet dea5cf8d544d4238bc4e90aa202fab00 (DEFAULT_TABLE table=test-table [id=3e1281bea5f84cd09dff937c9c631957]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:52:09.950085 11800 consensus_queue.cc:260] T dea5cf8d544d4238bc4e90aa202fab00 P 25f72b14a05d4ec6861e7a7bbbac9ef5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "982252a2a0f04384a24931f2215efa1e" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44087 } } peers { permanent_uuid: "74b028f8dcd8443493a06be381ff1317" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 44825 } } peers { permanent_uuid: "25f72b14a05d4ec6861e7a7bbbac9ef5" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 38397 } }
I20260808 16:52:09.950142 11059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dea5cf8d544d4238bc4e90aa202fab00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:52:09.950165 11800 ts_tablet_manager.cc:1428] T dea5cf8d544d4238bc4e90aa202fab00 P 25f72b14a05d4ec6861e7a7bbbac9ef5: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:09.950618 11798 tablet_bootstrap.cc:492] T dea5cf8d544d4238bc4e90aa202fab00 P 74b028f8dcd8443493a06be381ff1317: No bootstrap required, opened a new log
I20260808 16:52:09.950665 11798 ts_tablet_manager.cc:1397] T dea5cf8d544d4238bc4e90aa202fab00 P 74b028f8dcd8443493a06be381ff1317: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 16:52:09.950762 11798 raft_consensus.cc:348] T dea5cf8d544d4238bc4e90aa202fab00 P 74b028f8dcd8443493a06be381ff1317 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "982252a2a0f04384a24931f2215efa1e" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44087 } } peers { permanent_uuid: "74b028f8dcd8443493a06be381ff1317" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 44825 } } peers { permanent_uuid: "25f72b14a05d4ec6861e7a7bbbac9ef5" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 38397 } }
I20260808 16:52:09.950820 11798 raft_consensus.cc:374] T dea5cf8d544d4238bc4e90aa202fab00 P 74b028f8dcd8443493a06be381ff1317 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:52:09.950838 11798 raft_consensus.cc:729] T dea5cf8d544d4238bc4e90aa202fab00 P 74b028f8dcd8443493a06be381ff1317 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74b028f8dcd8443493a06be381ff1317, State: Initialized, Role: FOLLOWER
I20260808 16:52:09.950878 11798 consensus_queue.cc:260] T dea5cf8d544d4238bc4e90aa202fab00 P 74b028f8dcd8443493a06be381ff1317 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "982252a2a0f04384a24931f2215efa1e" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44087 } } peers { permanent_uuid: "74b028f8dcd8443493a06be381ff1317" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 44825 } } peers { permanent_uuid: "25f72b14a05d4ec6861e7a7bbbac9ef5" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 38397 } }
I20260808 16:52:09.950966 11798 ts_tablet_manager.cc:1428] T dea5cf8d544d4238bc4e90aa202fab00 P 74b028f8dcd8443493a06be381ff1317: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:09.951341 11799 tablet_bootstrap.cc:492] T dea5cf8d544d4238bc4e90aa202fab00 P 982252a2a0f04384a24931f2215efa1e: Bootstrap starting.
I20260808 16:52:09.951617 11799 tablet_bootstrap.cc:654] T dea5cf8d544d4238bc4e90aa202fab00 P 982252a2a0f04384a24931f2215efa1e: Neither blocks nor log segments found. Creating new log.
I20260808 16:52:09.952121 11799 tablet_bootstrap.cc:492] T dea5cf8d544d4238bc4e90aa202fab00 P 982252a2a0f04384a24931f2215efa1e: No bootstrap required, opened a new log
I20260808 16:52:09.952170 11799 ts_tablet_manager.cc:1397] T dea5cf8d544d4238bc4e90aa202fab00 P 982252a2a0f04384a24931f2215efa1e: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 16:52:09.952286 11799 raft_consensus.cc:348] T dea5cf8d544d4238bc4e90aa202fab00 P 982252a2a0f04384a24931f2215efa1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "982252a2a0f04384a24931f2215efa1e" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44087 } } peers { permanent_uuid: "74b028f8dcd8443493a06be381ff1317" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 44825 } } peers { permanent_uuid: "25f72b14a05d4ec6861e7a7bbbac9ef5" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 38397 } }
I20260808 16:52:09.952335 11799 raft_consensus.cc:374] T dea5cf8d544d4238bc4e90aa202fab00 P 982252a2a0f04384a24931f2215efa1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:52:09.952346 11799 raft_consensus.cc:729] T dea5cf8d544d4238bc4e90aa202fab00 P 982252a2a0f04384a24931f2215efa1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 982252a2a0f04384a24931f2215efa1e, State: Initialized, Role: FOLLOWER
I20260808 16:52:09.952373 11799 consensus_queue.cc:260] T dea5cf8d544d4238bc4e90aa202fab00 P 982252a2a0f04384a24931f2215efa1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "982252a2a0f04384a24931f2215efa1e" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44087 } } peers { permanent_uuid: "74b028f8dcd8443493a06be381ff1317" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 44825 } } peers { permanent_uuid: "25f72b14a05d4ec6861e7a7bbbac9ef5" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 38397 } }
I20260808 16:52:09.952420 11799 ts_tablet_manager.cc:1428] T dea5cf8d544d4238bc4e90aa202fab00 P 982252a2a0f04384a24931f2215efa1e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:10.007385 11759 raft_consensus.cc:482] T dea5cf8d544d4238bc4e90aa202fab00 P 982252a2a0f04384a24931f2215efa1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 16:52:10.007478 11759 raft_consensus.cc:504] T dea5cf8d544d4238bc4e90aa202fab00 P 982252a2a0f04384a24931f2215efa1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "982252a2a0f04384a24931f2215efa1e" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44087 } } peers { permanent_uuid: "74b028f8dcd8443493a06be381ff1317" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 44825 } } peers { permanent_uuid: "25f72b14a05d4ec6861e7a7bbbac9ef5" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 38397 } }
I20260808 16:52:10.007664 11759 leader_election.cc:288] T dea5cf8d544d4238bc4e90aa202fab00 P 982252a2a0f04384a24931f2215efa1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 74b028f8dcd8443493a06be381ff1317 (127.8.24.2:44825), 25f72b14a05d4ec6861e7a7bbbac9ef5 (127.8.24.3:38397)
I20260808 16:52:10.007819 11309 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dea5cf8d544d4238bc4e90aa202fab00" candidate_uuid: "982252a2a0f04384a24931f2215efa1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74b028f8dcd8443493a06be381ff1317" is_pre_election: true
I20260808 16:52:10.007819 11534 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dea5cf8d544d4238bc4e90aa202fab00" candidate_uuid: "982252a2a0f04384a24931f2215efa1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25f72b14a05d4ec6861e7a7bbbac9ef5" is_pre_election: true
I20260808 16:52:10.007915 11534 raft_consensus.cc:2454] T dea5cf8d544d4238bc4e90aa202fab00 P 25f72b14a05d4ec6861e7a7bbbac9ef5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 982252a2a0f04384a24931f2215efa1e in term 0.
I20260808 16:52:10.007915 11309 raft_consensus.cc:2454] T dea5cf8d544d4238bc4e90aa202fab00 P 74b028f8dcd8443493a06be381ff1317 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 982252a2a0f04384a24931f2215efa1e in term 0.
I20260808 16:52:10.008055 10995 leader_election.cc:302] T dea5cf8d544d4238bc4e90aa202fab00 P 982252a2a0f04384a24931f2215efa1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25f72b14a05d4ec6861e7a7bbbac9ef5, 982252a2a0f04384a24931f2215efa1e; no voters:
I20260808 16:52:10.008152 11759 raft_consensus.cc:2781] T dea5cf8d544d4238bc4e90aa202fab00 P 982252a2a0f04384a24931f2215efa1e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 16:52:10.008189 11759 raft_consensus.cc:482] T dea5cf8d544d4238bc4e90aa202fab00 P 982252a2a0f04384a24931f2215efa1e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 16:52:10.008206 11759 raft_consensus.cc:3037] T dea5cf8d544d4238bc4e90aa202fab00 P 982252a2a0f04384a24931f2215efa1e [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:52:10.008818 11759 raft_consensus.cc:504] T dea5cf8d544d4238bc4e90aa202fab00 P 982252a2a0f04384a24931f2215efa1e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "982252a2a0f04384a24931f2215efa1e" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44087 } } peers { permanent_uuid: "74b028f8dcd8443493a06be381ff1317" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 44825 } } peers { permanent_uuid: "25f72b14a05d4ec6861e7a7bbbac9ef5" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 38397 } }
I20260808 16:52:10.008925 11759 leader_election.cc:288] T dea5cf8d544d4238bc4e90aa202fab00 P 982252a2a0f04384a24931f2215efa1e [CANDIDATE]: Term 1 election: Requested vote from peers 74b028f8dcd8443493a06be381ff1317 (127.8.24.2:44825), 25f72b14a05d4ec6861e7a7bbbac9ef5 (127.8.24.3:38397)
I20260808 16:52:10.009105 11309 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dea5cf8d544d4238bc4e90aa202fab00" candidate_uuid: "982252a2a0f04384a24931f2215efa1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74b028f8dcd8443493a06be381ff1317"
I20260808 16:52:10.009135 11534 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dea5cf8d544d4238bc4e90aa202fab00" candidate_uuid: "982252a2a0f04384a24931f2215efa1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25f72b14a05d4ec6861e7a7bbbac9ef5"
I20260808 16:52:10.009179 11309 raft_consensus.cc:3037] T dea5cf8d544d4238bc4e90aa202fab00 P 74b028f8dcd8443493a06be381ff1317 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:52:10.009212 11534 raft_consensus.cc:3037] T dea5cf8d544d4238bc4e90aa202fab00 P 25f72b14a05d4ec6861e7a7bbbac9ef5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:52:10.009773 11309 raft_consensus.cc:2454] T dea5cf8d544d4238bc4e90aa202fab00 P 74b028f8dcd8443493a06be381ff1317 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 982252a2a0f04384a24931f2215efa1e in term 1.
I20260808 16:52:10.009967 10992 leader_election.cc:302] T dea5cf8d544d4238bc4e90aa202fab00 P 982252a2a0f04384a24931f2215efa1e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74b028f8dcd8443493a06be381ff1317, 982252a2a0f04384a24931f2215efa1e; no voters:
I20260808 16:52:10.010039 11534 raft_consensus.cc:2454] T dea5cf8d544d4238bc4e90aa202fab00 P 25f72b14a05d4ec6861e7a7bbbac9ef5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 982252a2a0f04384a24931f2215efa1e in term 1.
I20260808 16:52:10.010049 11759 raft_consensus.cc:2781] T dea5cf8d544d4238bc4e90aa202fab00 P 982252a2a0f04384a24931f2215efa1e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:52:10.010097 11759 raft_consensus.cc:686] T dea5cf8d544d4238bc4e90aa202fab00 P 982252a2a0f04384a24931f2215efa1e [term 1 LEADER]: Becoming Leader. State: Replica: 982252a2a0f04384a24931f2215efa1e, State: Running, Role: LEADER
I20260808 16:52:10.010139 11759 consensus_queue.cc:237] T dea5cf8d544d4238bc4e90aa202fab00 P 982252a2a0f04384a24931f2215efa1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "982252a2a0f04384a24931f2215efa1e" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44087 } } peers { permanent_uuid: "74b028f8dcd8443493a06be381ff1317" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 44825 } } peers { permanent_uuid: "25f72b14a05d4ec6861e7a7bbbac9ef5" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 38397 } }
I20260808 16:52:10.010656 10856 catalog_manager.cc:5495] T dea5cf8d544d4238bc4e90aa202fab00 P 982252a2a0f04384a24931f2215efa1e reported cstate change: term changed from 0 to 1, leader changed from <none> to 982252a2a0f04384a24931f2215efa1e (127.8.24.1). New cstate: current_term: 1 leader_uuid: "982252a2a0f04384a24931f2215efa1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "982252a2a0f04384a24931f2215efa1e" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44087 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74b028f8dcd8443493a06be381ff1317" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 44825 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25f72b14a05d4ec6861e7a7bbbac9ef5" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 38397 } health_report { overall_health: UNKNOWN } } }
I20260808 16:52:10.025400 11534 raft_consensus.cc:1261] T dea5cf8d544d4238bc4e90aa202fab00 P 25f72b14a05d4ec6861e7a7bbbac9ef5 [term 1 FOLLOWER]: Refusing update from remote peer 982252a2a0f04384a24931f2215efa1e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:52:10.025461 11309 raft_consensus.cc:1261] T dea5cf8d544d4238bc4e90aa202fab00 P 74b028f8dcd8443493a06be381ff1317 [term 1 FOLLOWER]: Refusing update from remote peer 982252a2a0f04384a24931f2215efa1e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:52:10.025630 11742 consensus_queue.cc:1035] T dea5cf8d544d4238bc4e90aa202fab00 P 982252a2a0f04384a24931f2215efa1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "25f72b14a05d4ec6861e7a7bbbac9ef5" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 38397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:52:10.025723 11751 consensus_queue.cc:1035] T dea5cf8d544d4238bc4e90aa202fab00 P 982252a2a0f04384a24931f2215efa1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "74b028f8dcd8443493a06be381ff1317" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 44825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 16:52:10.031265 11748 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260808 16:52:11.549448 10855 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59240:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 16:52:11.549547 10855 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59240:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 16:52:11.550352 10855 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5627ffa85f6546a1a64643aa34ab1e6f: Sending DeleteTablet for 3 replicas of tablet dea5cf8d544d4238bc4e90aa202fab00
I20260808 16:52:11.550583 11059 tablet_service.cc:1514] Processing DeleteTablet for tablet dea5cf8d544d4238bc4e90aa202fab00 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:52:11 UTC) from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:43148
I20260808 16:52:11.550649 11278 tablet_service.cc:1514] Processing DeleteTablet for tablet dea5cf8d544d4238bc4e90aa202fab00 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:52:11 UTC) from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:51682
I20260808 16:52:11.550735 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 10971
I20260808 16:52:11.551193 11501 tablet_service.cc:1514] Processing DeleteTablet for tablet dea5cf8d544d4238bc4e90aa202fab00 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:52:11 UTC) from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:50978
I20260808 16:52:11.558524 11868 tablet_replica.cc:330] T dea5cf8d544d4238bc4e90aa202fab00 P 74b028f8dcd8443493a06be381ff1317: stopping tablet replica
I20260808 16:52:11.558645 11868 raft_consensus.cc:2229] T dea5cf8d544d4238bc4e90aa202fab00 P 74b028f8dcd8443493a06be381ff1317 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 16:52:11.558732 11868 raft_consensus.cc:2258] T dea5cf8d544d4238bc4e90aa202fab00 P 74b028f8dcd8443493a06be381ff1317 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:52:11.560184 11868 ts_tablet_manager.cc:1905] T dea5cf8d544d4238bc4e90aa202fab00 P 74b028f8dcd8443493a06be381ff1317: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:52:11.560511 11869 tablet_replica.cc:330] T dea5cf8d544d4238bc4e90aa202fab00 P 25f72b14a05d4ec6861e7a7bbbac9ef5: stopping tablet replica
I20260808 16:52:11.560611 11869 raft_consensus.cc:2229] T dea5cf8d544d4238bc4e90aa202fab00 P 25f72b14a05d4ec6861e7a7bbbac9ef5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 16:52:11.560658 11869 raft_consensus.cc:2258] T dea5cf8d544d4238bc4e90aa202fab00 P 25f72b14a05d4ec6861e7a7bbbac9ef5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:52:11.560892 11869 ts_tablet_manager.cc:1905] T dea5cf8d544d4238bc4e90aa202fab00 P 25f72b14a05d4ec6861e7a7bbbac9ef5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:52:11.561878 11868 ts_tablet_manager.cc:1918] T dea5cf8d544d4238bc4e90aa202fab00 P 74b028f8dcd8443493a06be381ff1317: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 16:52:11.561954 11868 log.cc:1196] T dea5cf8d544d4238bc4e90aa202fab00 P 74b028f8dcd8443493a06be381ff1317: Deleting WAL directory at /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/minicluster-data/ts-1/wal/wals/dea5cf8d544d4238bc4e90aa202fab00
I20260808 16:52:11.562202 11869 ts_tablet_manager.cc:1918] T dea5cf8d544d4238bc4e90aa202fab00 P 25f72b14a05d4ec6861e7a7bbbac9ef5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 16:52:11.562235 11868 ts_tablet_manager.cc:1939] T dea5cf8d544d4238bc4e90aa202fab00 P 74b028f8dcd8443493a06be381ff1317: Deleting consensus metadata
I20260808 16:52:11.562264 11869 log.cc:1196] T dea5cf8d544d4238bc4e90aa202fab00 P 25f72b14a05d4ec6861e7a7bbbac9ef5: Deleting WAL directory at /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786207919384338-8288-0/minicluster-data/ts-2/wal/wals/dea5cf8d544d4238bc4e90aa202fab00
I20260808 16:52:11.562589 11869 ts_tablet_manager.cc:1939] T dea5cf8d544d4238bc4e90aa202fab00 P 25f72b14a05d4ec6861e7a7bbbac9ef5: Deleting consensus metadata
I20260808 16:52:11.562585 10837 catalog_manager.cc:4841] TS 74b028f8dcd8443493a06be381ff1317 (127.8.24.2:44825): tablet dea5cf8d544d4238bc4e90aa202fab00 (table test-table [id=3e1281bea5f84cd09dff937c9c631957]) successfully deleted
I20260808 16:52:11.562942 10841 catalog_manager.cc:4841] TS 25f72b14a05d4ec6861e7a7bbbac9ef5 (127.8.24.3:38397): tablet dea5cf8d544d4238bc4e90aa202fab00 (table test-table [id=3e1281bea5f84cd09dff937c9c631957]) successfully deleted
W20260808 16:52:11.567560 10840 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.24.1:44087 (error 108)
I20260808 16:52:11.567766 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 11179
I20260808 16:52:11.571087 11871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:11.567881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:44087 (local address 127.0.0.1:43172)
0808 16:52:11.570999 (+ 3118us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:11.571050 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.1:44087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3092,"thread_start_us":56,"threads_started":1}
W20260808 16:52:11.571180 10840 catalog_manager.cc:4584] TS 982252a2a0f04384a24931f2215efa1e (127.8.24.1:44087): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet dea5cf8d544d4238bc4e90aa202fab00: Network error: Client connection negotiation failed: client connection to 127.8.24.1:44087: connect: Connection refused (error 111)
I20260808 16:52:11.578781 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 11410
I20260808 16:52:11.585208 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 10815
2026-08-08T16:52:11Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (5919 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786207919384338-8288-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 16:52:11 dist-test-slave-6zbq krb5kdc[11874](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 16:52:11 dist-test-slave-6zbq krb5kdc[11874](info): set up 2 sockets
Aug 08 16:52:11 dist-test-slave-6zbq krb5kdc[11874](info): commencing operation
krb5kdc: starting...
W20260808 16:52:12.729264 8288 mini_kdc.cc:121] Time spent starting KDC: real 1.120s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786207919384338-8288-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786207919384338-8288-0/krb5kdc/test-user.keytab.
Aug 08 16:52:12 dist-test-slave-6zbq krb5kdc[11874](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207932, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T16:52:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T16:52:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.24.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
I20260808 16:52:12.834995 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786207919384338-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786207919384338-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786207919384338-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:43237
--webserver_interface=127.8.24.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:33791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:43237
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:52:12.913571 11894 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:52:12.913748 11894 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:52:12.913767 11894 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:52:12.914853 11894 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 16:52:12.914891 11894 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:52:12.914903 11894 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:52:12.914914 11894 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 16:52:12.914928 11894 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 16:52:12.916299 11894 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:33791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786207919384338-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786207919384338-8288-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:43237
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:43237
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.24.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11894
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786207919384338-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:52:12.916476 11894 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:52:12.916633 11894 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 16:52:12.920955 11894 server_base.cc:1034] running on GCE node
W20260808 16:52:12.920933 11899 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:12.920930 11902 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:12.920924 11900 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:12.921360 11894 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:52:12.921607 11894 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:52:12.922734 11894 hybrid_clock.cc:648] HybridClock initialized: now 1786207932922719 us; error 31 us; skew 500 ppm
Aug 08 16:52:12 dist-test-slave-6zbq krb5kdc[11874](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207932, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:52:12.924896 11894 init.cc:377] Logged in from keytab as kudu/127.8.24.62@KRBTEST.COM (short username kudu)
I20260808 16:52:12.925279 11894 webserver.cc:460] Webserver started at http://127.8.24.62:45301/ using document root <none> and password file <none>
I20260808 16:52:12.925459 11894 fs_manager.cc:359] Metadata directory not provided
I20260808 16:52:12.925500 11894 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:52:12.925616 11894 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:52:12.926604 11894 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786207919384338-8288-0/minicluster-data/master-0/data/instance:
uuid: "e47eeba69ab64e4893b4d409e59c3402"
format_stamp: "Formatted at 2026-08-08 16:52:12 on dist-test-slave-6zbq"
server_key: "ba84b866f62e486f3abed6e952810c78"
server_key_iv: "53d71f4a1735407cd69cf125c995d091"
server_key_version: "encryptionkey@0"
I20260808 16:52:12.926919 11894 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786207919384338-8288-0/minicluster-data/master-0/wal/instance:
uuid: "e47eeba69ab64e4893b4d409e59c3402"
format_stamp: "Formatted at 2026-08-08 16:52:12 on dist-test-slave-6zbq"
server_key: "ba84b866f62e486f3abed6e952810c78"
server_key_iv: "53d71f4a1735407cd69cf125c995d091"
server_key_version: "encryptionkey@0"
I20260808 16:52:12.928061 11894 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 16:52:12.928711 11917 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:12.928860 11894 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 16:52:12.928937 11894 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786207919384338-8288-0/minicluster-data/master-0/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786207919384338-8288-0/minicluster-data/master-0/wal
uuid: "e47eeba69ab64e4893b4d409e59c3402"
format_stamp: "Formatted at 2026-08-08 16:52:12 on dist-test-slave-6zbq"
server_key: "ba84b866f62e486f3abed6e952810c78"
server_key_iv: "53d71f4a1735407cd69cf125c995d091"
server_key_version: "encryptionkey@0"
I20260808 16:52:12.929001 11894 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786207919384338-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786207919384338-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786207919384338-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:52:12.946492 11894 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:52:12.948441 11894 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:52:12.948560 11894 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:52:12.952096 11894 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.62:43237
I20260808 16:52:12.952128 12000 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.62:43237 every 8 connection(s)
I20260808 16:52:12.952420 11894 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
I20260808 16:52:12.953158 12001 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:52:12.955252 12001 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e47eeba69ab64e4893b4d409e59c3402: Bootstrap starting.
I20260808 16:52:12.955780 12001 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e47eeba69ab64e4893b4d409e59c3402: Neither blocks nor log segments found. Creating new log.
I20260808 16:52:12.956033 12001 log.cc:824] T 00000000000000000000000000000000 P e47eeba69ab64e4893b4d409e59c3402: Log is configured to *not* fsync() on all Append() calls
I20260808 16:52:12.956619 12001 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e47eeba69ab64e4893b4d409e59c3402: No bootstrap required, opened a new log
I20260808 16:52:12.958127 12001 raft_consensus.cc:348] T 00000000000000000000000000000000 P e47eeba69ab64e4893b4d409e59c3402 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e47eeba69ab64e4893b4d409e59c3402" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 43237 } }
I20260808 16:52:12.958252 12001 raft_consensus.cc:374] T 00000000000000000000000000000000 P e47eeba69ab64e4893b4d409e59c3402 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:52:12.958300 12001 raft_consensus.cc:729] T 00000000000000000000000000000000 P e47eeba69ab64e4893b4d409e59c3402 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e47eeba69ab64e4893b4d409e59c3402, State: Initialized, Role: FOLLOWER
I20260808 16:52:12.958456 12001 consensus_queue.cc:260] T 00000000000000000000000000000000 P e47eeba69ab64e4893b4d409e59c3402 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e47eeba69ab64e4893b4d409e59c3402" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 43237 } }
I20260808 16:52:12.958521 12001 raft_consensus.cc:388] T 00000000000000000000000000000000 P e47eeba69ab64e4893b4d409e59c3402 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 16:52:12.958544 12001 raft_consensus.cc:482] T 00000000000000000000000000000000 P e47eeba69ab64e4893b4d409e59c3402 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 16:52:12.958587 12001 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e47eeba69ab64e4893b4d409e59c3402 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:52:12.958942 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 11894
I20260808 16:52:12.959034 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786207919384338-8288-0/minicluster-data/master-0/wal/instance
I20260808 16:52:12.959154 12001 raft_consensus.cc:504] T 00000000000000000000000000000000 P e47eeba69ab64e4893b4d409e59c3402 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e47eeba69ab64e4893b4d409e59c3402" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 43237 } }
I20260808 16:52:12.959206 8288 external_mini_cluster.cc:1428] Setting key 90ae924cdc0462451094fcc378ab2652
I20260808 16:52:12.959254 12001 leader_election.cc:302] T 00000000000000000000000000000000 P e47eeba69ab64e4893b4d409e59c3402 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e47eeba69ab64e4893b4d409e59c3402; no voters:
I20260808 16:52:12.959422 12001 leader_election.cc:288] T 00000000000000000000000000000000 P e47eeba69ab64e4893b4d409e59c3402 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 16:52:12.959455 12004 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e47eeba69ab64e4893b4d409e59c3402 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:52:12.959661 12001 sys_catalog.cc:564] T 00000000000000000000000000000000 P e47eeba69ab64e4893b4d409e59c3402 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 16:52:12.959880 12004 raft_consensus.cc:686] T 00000000000000000000000000000000 P e47eeba69ab64e4893b4d409e59c3402 [term 1 LEADER]: Becoming Leader. State: Replica: e47eeba69ab64e4893b4d409e59c3402, State: Running, Role: LEADER
I20260808 16:52:12.960031 12004 consensus_queue.cc:237] T 00000000000000000000000000000000 P e47eeba69ab64e4893b4d409e59c3402 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e47eeba69ab64e4893b4d409e59c3402" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 43237 } }
I20260808 16:52:12.960386 12004 sys_catalog.cc:455] T 00000000000000000000000000000000 P e47eeba69ab64e4893b4d409e59c3402 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e47eeba69ab64e4893b4d409e59c3402. Latest consensus state: current_term: 1 leader_uuid: "e47eeba69ab64e4893b4d409e59c3402" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e47eeba69ab64e4893b4d409e59c3402" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 43237 } } }
I20260808 16:52:12.960517 12004 sys_catalog.cc:458] T 00000000000000000000000000000000 P e47eeba69ab64e4893b4d409e59c3402 [sys.catalog]: This master's current role is: LEADER
I20260808 16:52:12.961001 12022 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 16:52:12.961222 12010 sys_catalog.cc:455] T 00000000000000000000000000000000 P e47eeba69ab64e4893b4d409e59c3402 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e47eeba69ab64e4893b4d409e59c3402" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e47eeba69ab64e4893b4d409e59c3402" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 43237 } } }
I20260808 16:52:12.961555 12010 sys_catalog.cc:458] T 00000000000000000000000000000000 P e47eeba69ab64e4893b4d409e59c3402 [sys.catalog]: This master's current role is: LEADER
I20260808 16:52:12.961591 12022 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 16:52:12.963670 12022 catalog_manager.cc:1347] Generated new cluster ID: c991566448d74e1b86ba0efd875fb365
I20260808 16:52:12.963722 12022 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 16:52:12 dist-test-slave-6zbq krb5kdc[11874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207932, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260808 16:52:12.970484 12007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:12.959605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50448 (local address 127.8.24.62:43237)
0808 16:52:12.959864 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:12.959873 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:12.959911 (+ 38us) server_negotiation.cc:407] Connection header received
0808 16:52:12.960017 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:12.960024 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:12.960083 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:12.960240 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:12.960964 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:12.962251 (+ 1287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:12.963305 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:12.963754 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:12.963861 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:12.966480 (+ 2619us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:12.966484 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:12.966484 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:12.966496 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:12.968944 (+ 2448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:12.969298 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:12.969301 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:12.969301 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:12.969424 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:12.969596 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:12.969597 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:12.969597 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:12.969885 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:12.970042 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:12.970107 (+ 65us) server_negotiation.cc:299] Negotiation successful
0808 16:52:12.970240 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":210,"thread_start_us":81,"threads_started":1}
I20260808 16:52:12.982471 12022 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 16:52:12.982918 12022 catalog_manager.cc:1504] Loading token signing keys...
I20260808 16:52:12.990150 12022 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e47eeba69ab64e4893b4d409e59c3402: Generated new TSK 0
I20260808 16:52:12.990304 12022 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.24.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.1.keytab.
Entry for principal kudu/127.8.24.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.1.keytab.
I20260808 16:52:13.016250 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786207919384338-8288-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786207919384338-8288-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786207919384338-8288-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786207919384338-8288-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.1:0
--local_ip_for_outbound_sockets=127.8.24.1
--webserver_interface=127.8.24.1
--webserver_port=0
--tserver_master_addrs=127.8.24.62:43237
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.1.keytab
--principal=kudu/127.8.24.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:33791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:52:13.112870 12035 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:52:13.113058 12035 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:52:13.113087 12035 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:52:13.114189 12035 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:52:13.114229 12035 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:52:13.114272 12035 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.1
I20260808 16:52:13.115682 12035 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:33791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786207919384338-8288-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786207919384338-8288-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.1.keytab
--principal=kudu/127.8.24.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786207919384338-8288-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.24.1
--webserver_port=0
--tserver_master_addrs=127.8.24.62:43237
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12035
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.1
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786207919384338-8288-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:52:13.115911 12035 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:52:13.116109 12035 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:52:13.120784 12044 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:13.120800 12040 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:13.120792 12042 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:13.120792 12035 server_base.cc:1034] running on GCE node
I20260808 16:52:13.121134 12035 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:52:13.121374 12035 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:52:13.122507 12035 hybrid_clock.cc:648] HybridClock initialized: now 1786207933122496 us; error 29 us; skew 500 ppm
Aug 08 16:52:13 dist-test-slave-6zbq krb5kdc[11874](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207933, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:52:13.124778 12035 init.cc:377] Logged in from keytab as kudu/127.8.24.1@KRBTEST.COM (short username kudu)
I20260808 16:52:13.125181 12035 webserver.cc:460] Webserver started at http://127.8.24.1:39587/ using document root <none> and password file <none>
I20260808 16:52:13.125370 12035 fs_manager.cc:359] Metadata directory not provided
I20260808 16:52:13.125407 12035 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:52:13.125496 12035 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:52:13.126480 12035 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786207919384338-8288-0/minicluster-data/ts-0/data/instance:
uuid: "6374ebbbb625499c99e9a84f618ff7f4"
format_stamp: "Formatted at 2026-08-08 16:52:13 on dist-test-slave-6zbq"
server_key: "7427df492dc0b7cf2587ff33c9947660"
server_key_iv: "285d35bf12d7f4b94a655b5e222041c3"
server_key_version: "encryptionkey@0"
I20260808 16:52:13.126771 12035 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786207919384338-8288-0/minicluster-data/ts-0/wal/instance:
uuid: "6374ebbbb625499c99e9a84f618ff7f4"
format_stamp: "Formatted at 2026-08-08 16:52:13 on dist-test-slave-6zbq"
server_key: "7427df492dc0b7cf2587ff33c9947660"
server_key_iv: "285d35bf12d7f4b94a655b5e222041c3"
server_key_version: "encryptionkey@0"
I20260808 16:52:13.127978 12035 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 16:52:13.128592 12058 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:13.128754 12035 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 16:52:13.128834 12035 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786207919384338-8288-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786207919384338-8288-0/minicluster-data/ts-0/wal
uuid: "6374ebbbb625499c99e9a84f618ff7f4"
format_stamp: "Formatted at 2026-08-08 16:52:13 on dist-test-slave-6zbq"
server_key: "7427df492dc0b7cf2587ff33c9947660"
server_key_iv: "285d35bf12d7f4b94a655b5e222041c3"
server_key_version: "encryptionkey@0"
I20260808 16:52:13.128885 12035 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786207919384338-8288-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786207919384338-8288-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786207919384338-8288-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:52:13.150647 12035 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:52:13.152202 12035 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:52:13.152321 12035 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:52:13.152513 12035 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:52:13.152781 12035 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:52:13.152813 12035 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:13.152833 12035 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:52:13.152858 12035 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:13.157995 12035 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.1:33695
I20260808 16:52:13.158031 12228 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.1:33695 every 8 connection(s)
I20260808 16:52:13.158301 12035 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786207919384338-8288-0/minicluster-data/ts-0/data/info.pb
I20260808 16:52:13.161021 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 12035
I20260808 16:52:13.161125 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786207919384338-8288-0/minicluster-data/ts-0/wal/instance
I20260808 16:52:13.161281 8288 external_mini_cluster.cc:1428] Setting key 5e0df56307ea9de50fadd519e3be5c4a
Aug 08 16:52:13 dist-test-slave-6zbq krb5kdc[11874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207933, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.1@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260808 16:52:13.167456 12007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:13.158902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:40581 (local address 127.8.24.62:43237)
0808 16:52:13.158959 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:13.158961 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:13.159184 (+ 223us) server_negotiation.cc:407] Connection header received
0808 16:52:13.159642 (+ 458us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:13.159643 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:13.159660 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:13.159709 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:13.160615 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:13.161313 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:13.162540 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:13.162804 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:13.162875 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:13.164946 (+ 2071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:13.164948 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:13.164948 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:13.164958 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:13.166197 (+ 1239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:13.166592 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:13.166593 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:13.166593 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:13.166649 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:13.166865 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:13.166866 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:13.166866 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:13.166975 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:13.167031 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:13.167281 (+ 250us) server_negotiation.cc:299] Negotiation successful
0808 16:52:13.167368 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 16:52:13.167573 12234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:13.158877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:43237 (local address 127.8.24.1:40581)
0808 16:52:13.159029 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:13.159091 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:13.159547 (+ 456us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:13.159785 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:13.159788 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:13.160353 (+ 565us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:13.160527 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:13.160532 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:13.161429 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:13.161430 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:13.162459 (+ 1029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:13.162462 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:13.162876 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:13.162877 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:13.162972 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:13.163409 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:13.163417 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:13.164863 (+ 1446us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:52:13.166265 (+ 1402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:13.166273 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:13.166275 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:13.166520 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:13.166699 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:13.166700 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:13.166701 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:13.166809 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:13.167030 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:52:13.167036 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:52:13.167160 (+ 124us) client_negotiation.cc:769] Sending connection context
0808 16:52:13.167244 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 16:52:13.167383 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":78,"threads_started":1}
I20260808 16:52:13.168022 12232 heartbeater.cc:347] Connected to a master server at 127.8.24.62:43237
I20260808 16:52:13.168120 12232 heartbeater.cc:464] Registering TS with master...
I20260808 16:52:13.168300 12232 heartbeater.cc:511] Master 127.8.24.62:43237 requested a full tablet report, sending...
I20260808 16:52:13.168797 11943 ts_manager.cc:194] Registered new tserver with Master: 6374ebbbb625499c99e9a84f618ff7f4 (127.8.24.1:33695)
I20260808 16:52:13.169366 11943 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.1@KRBTEST.COM'} at 127.8.24.1:40581
WARNING: no policy specified for kudu/127.8.24.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.2.keytab.
Entry for principal kudu/127.8.24.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.2.keytab.
I20260808 16:52:13.183293 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786207919384338-8288-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786207919384338-8288-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786207919384338-8288-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786207919384338-8288-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.2:0
--local_ip_for_outbound_sockets=127.8.24.2
--webserver_interface=127.8.24.2
--webserver_port=0
--tserver_master_addrs=127.8.24.62:43237
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.2.keytab
--principal=kudu/127.8.24.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:33791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:52:13.260695 12237 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:52:13.260888 12237 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:52:13.260918 12237 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:52:13.262029 12237 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:52:13.262070 12237 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:52:13.262104 12237 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.2
I20260808 16:52:13.263494 12237 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:33791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786207919384338-8288-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786207919384338-8288-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.2.keytab
--principal=kudu/127.8.24.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786207919384338-8288-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.24.2
--webserver_port=0
--tserver_master_addrs=127.8.24.62:43237
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12237
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.2
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786207919384338-8288-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:52:13.263703 12237 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:52:13.263904 12237 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:52:13.268316 12248 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:13.268343 12251 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:13.268392 12249 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:13.268494 12237 server_base.cc:1034] running on GCE node
I20260808 16:52:13.268679 12237 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:52:13.268898 12237 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:52:13.270030 12237 hybrid_clock.cc:648] HybridClock initialized: now 1786207933270016 us; error 29 us; skew 500 ppm
Aug 08 16:52:13 dist-test-slave-6zbq krb5kdc[11874](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207933, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:52:13.272344 12237 init.cc:377] Logged in from keytab as kudu/127.8.24.2@KRBTEST.COM (short username kudu)
I20260808 16:52:13.272733 12237 webserver.cc:460] Webserver started at http://127.8.24.2:34295/ using document root <none> and password file <none>
I20260808 16:52:13.272918 12237 fs_manager.cc:359] Metadata directory not provided
I20260808 16:52:13.272964 12237 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:52:13.273083 12237 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:52:13.274012 12237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786207919384338-8288-0/minicluster-data/ts-1/data/instance:
uuid: "b196e3ab28664b1085fec7889680dc09"
format_stamp: "Formatted at 2026-08-08 16:52:13 on dist-test-slave-6zbq"
server_key: "fcfd1f421e0248d549a74f58661dbbe2"
server_key_iv: "1e6cf2e2acd5f477b22e04a791a628b9"
server_key_version: "encryptionkey@0"
I20260808 16:52:13.274335 12237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786207919384338-8288-0/minicluster-data/ts-1/wal/instance:
uuid: "b196e3ab28664b1085fec7889680dc09"
format_stamp: "Formatted at 2026-08-08 16:52:13 on dist-test-slave-6zbq"
server_key: "fcfd1f421e0248d549a74f58661dbbe2"
server_key_iv: "1e6cf2e2acd5f477b22e04a791a628b9"
server_key_version: "encryptionkey@0"
I20260808 16:52:13.275530 12237 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 16:52:13.276154 12261 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:13.276307 12237 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:13.276386 12237 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786207919384338-8288-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786207919384338-8288-0/minicluster-data/ts-1/wal
uuid: "b196e3ab28664b1085fec7889680dc09"
format_stamp: "Formatted at 2026-08-08 16:52:13 on dist-test-slave-6zbq"
server_key: "fcfd1f421e0248d549a74f58661dbbe2"
server_key_iv: "1e6cf2e2acd5f477b22e04a791a628b9"
server_key_version: "encryptionkey@0"
I20260808 16:52:13.276444 12237 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786207919384338-8288-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786207919384338-8288-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786207919384338-8288-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:52:13.284265 12237 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:52:13.286166 12237 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:52:13.286296 12237 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:52:13.286499 12237 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:52:13.286759 12237 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:52:13.286798 12237 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:13.286823 12237 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:52:13.286839 12237 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:13.292057 12237 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.2:45665
I20260808 16:52:13.292091 12442 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.2:45665 every 8 connection(s)
I20260808 16:52:13.292357 12237 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786207919384338-8288-0/minicluster-data/ts-1/data/info.pb
I20260808 16:52:13.297043 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 12237
I20260808 16:52:13.297137 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786207919384338-8288-0/minicluster-data/ts-1/wal/instance
I20260808 16:52:13.297322 8288 external_mini_cluster.cc:1428] Setting key d6d73568342862ff638d65724c3791c8
Aug 08 16:52:13 dist-test-slave-6zbq krb5kdc[11874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207933, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.2@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260808 16:52:13.300305 12007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:13.292955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:50283 (local address 127.8.24.62:43237)
0808 16:52:13.293013 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:13.293016 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:13.293163 (+ 147us) server_negotiation.cc:407] Connection header received
0808 16:52:13.293585 (+ 422us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:13.293585 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:13.293595 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:13.293644 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:13.294512 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:13.294998 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:13.295885 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:13.296055 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:13.296096 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:13.297985 (+ 1889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:13.297986 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:13.297986 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:13.297994 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:13.299131 (+ 1137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:13.299483 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:13.299484 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:13.299484 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:13.299535 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:13.299737 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:13.299738 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:13.299738 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:13.299843 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:13.299899 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:13.300139 (+ 240us) server_negotiation.cc:299] Negotiation successful
0808 16:52:13.300219 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 16:52:13.300446 12447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:13.292901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:43237 (local address 127.8.24.2:50283)
0808 16:52:13.293048 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:13.293084 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:13.293501 (+ 417us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:13.293705 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:13.293708 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:13.294216 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:13.294444 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:13.294449 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:13.295057 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:13.295058 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:13.295827 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:13.295830 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:13.296105 (+ 275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:13.296106 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:13.296191 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:13.296534 (+ 343us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:13.296541 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:13.297898 (+ 1357us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:52:13.299203 (+ 1305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:13.299209 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:13.299211 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:13.299417 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:13.299589 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:13.299590 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:13.299590 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:13.299682 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:13.299900 (+ 218us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:52:13.299906 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:52:13.300026 (+ 120us) client_negotiation.cc:769] Sending connection context
0808 16:52:13.300108 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 16:52:13.300246 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":77,"threads_started":1}
I20260808 16:52:13.300861 12444 heartbeater.cc:347] Connected to a master server at 127.8.24.62:43237
I20260808 16:52:13.300954 12444 heartbeater.cc:464] Registering TS with master...
I20260808 16:52:13.301124 12444 heartbeater.cc:511] Master 127.8.24.62:43237 requested a full tablet report, sending...
I20260808 16:52:13.301466 11943 ts_manager.cc:194] Registered new tserver with Master: b196e3ab28664b1085fec7889680dc09 (127.8.24.2:45665)
I20260808 16:52:13.301921 11943 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.2@KRBTEST.COM'} at 127.8.24.2:50283
WARNING: no policy specified for kudu/127.8.24.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.3.keytab.
Entry for principal kudu/127.8.24.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.3.keytab.
I20260808 16:52:13.320050 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786207919384338-8288-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786207919384338-8288-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786207919384338-8288-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786207919384338-8288-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.3:0
--local_ip_for_outbound_sockets=127.8.24.3
--webserver_interface=127.8.24.3
--webserver_port=0
--tserver_master_addrs=127.8.24.62:43237
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.3.keytab
--principal=kudu/127.8.24.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:33791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:52:13.397375 12452 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:52:13.397550 12452 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:52:13.397571 12452 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:52:13.398658 12452 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:52:13.398692 12452 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:52:13.398716 12452 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.3
I20260808 16:52:13.400040 12452 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:33791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786207919384338-8288-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786207919384338-8288-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.3.keytab
--principal=kudu/127.8.24.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786207919384338-8288-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.24.3
--webserver_port=0
--tserver_master_addrs=127.8.24.62:43237
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12452
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.3
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786207919384338-8288-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:52:13.400206 12452 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:52:13.400374 12452 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:52:13.404778 12459 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:13.404822 12458 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:13.404860 12452 server_base.cc:1034] running on GCE node
W20260808 16:52:13.404822 12463 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:13.405156 12452 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:52:13.405346 12452 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:52:13.406471 12452 hybrid_clock.cc:648] HybridClock initialized: now 1786207933406460 us; error 30 us; skew 500 ppm
Aug 08 16:52:13 dist-test-slave-6zbq krb5kdc[11874](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207933, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:52:13.408622 12452 init.cc:377] Logged in from keytab as kudu/127.8.24.3@KRBTEST.COM (short username kudu)
I20260808 16:52:13.409009 12452 webserver.cc:460] Webserver started at http://127.8.24.3:39677/ using document root <none> and password file <none>
I20260808 16:52:13.409193 12452 fs_manager.cc:359] Metadata directory not provided
I20260808 16:52:13.409235 12452 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:52:13.409346 12452 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:52:13.410250 12452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786207919384338-8288-0/minicluster-data/ts-2/data/instance:
uuid: "af049643d76c4b96b39eb51de88e6270"
format_stamp: "Formatted at 2026-08-08 16:52:13 on dist-test-slave-6zbq"
server_key: "b1ac10737f233a974e898f0c4e2f8e39"
server_key_iv: "07de52338304bfb4d5bba94476342044"
server_key_version: "encryptionkey@0"
I20260808 16:52:13.410594 12452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786207919384338-8288-0/minicluster-data/ts-2/wal/instance:
uuid: "af049643d76c4b96b39eb51de88e6270"
format_stamp: "Formatted at 2026-08-08 16:52:13 on dist-test-slave-6zbq"
server_key: "b1ac10737f233a974e898f0c4e2f8e39"
server_key_iv: "07de52338304bfb4d5bba94476342044"
server_key_version: "encryptionkey@0"
I20260808 16:52:13.411715 12452 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 16:52:13.412330 12475 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:13.412464 12452 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 16:52:13.412537 12452 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786207919384338-8288-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786207919384338-8288-0/minicluster-data/ts-2/wal
uuid: "af049643d76c4b96b39eb51de88e6270"
format_stamp: "Formatted at 2026-08-08 16:52:13 on dist-test-slave-6zbq"
server_key: "b1ac10737f233a974e898f0c4e2f8e39"
server_key_iv: "07de52338304bfb4d5bba94476342044"
server_key_version: "encryptionkey@0"
I20260808 16:52:13.412598 12452 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786207919384338-8288-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786207919384338-8288-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786207919384338-8288-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:52:13.422600 12452 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:52:13.424090 12452 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:52:13.424201 12452 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:52:13.424382 12452 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:52:13.424639 12452 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:52:13.424676 12452 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:13.424695 12452 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:52:13.424718 12452 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:13.430033 12452 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.3:36957
I20260808 16:52:13.430071 12644 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.3:36957 every 8 connection(s)
I20260808 16:52:13.430313 12452 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786207919384338-8288-0/minicluster-data/ts-2/data/info.pb
I20260808 16:52:13.434010 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 12452
I20260808 16:52:13.434098 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786207919384338-8288-0/minicluster-data/ts-2/wal/instance
I20260808 16:52:13.434264 8288 external_mini_cluster.cc:1428] Setting key 9b863a59550910bd64a3a5266405a413
Aug 08 16:52:13 dist-test-slave-6zbq krb5kdc[11874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207933, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.3@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260808 16:52:13.438515 12007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:13.430917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:43297 (local address 127.8.24.62:43237)
0808 16:52:13.430980 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:13.430982 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:13.431123 (+ 141us) server_negotiation.cc:407] Connection header received
0808 16:52:13.431601 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:13.431603 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:13.431617 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:13.431666 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:13.432518 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:13.433009 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:13.433934 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:13.434163 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:13.434216 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:13.436084 (+ 1868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:13.436086 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:13.436086 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:13.436094 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:13.437297 (+ 1203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:13.437640 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:13.437641 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:13.437641 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:13.437697 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:13.437907 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:13.437908 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:13.437908 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:13.438017 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:13.438072 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:13.438320 (+ 248us) server_negotiation.cc:299] Negotiation successful
0808 16:52:13.438429 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 16:52:13.438640 12648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:13.430871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:43237 (local address 127.8.24.3:43297)
0808 16:52:13.431022 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:13.431053 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:13.431514 (+ 461us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:13.431726 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:13.431731 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:13.432246 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:13.432415 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:13.432419 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:13.433090 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:13.433091 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:13.433838 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:13.433840 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:13.434237 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:13.434239 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:13.434378 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:13.434802 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:13.434810 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:13.436000 (+ 1190us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:52:13.437370 (+ 1370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:13.437376 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:13.437378 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:13.437576 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:13.437748 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:13.437749 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:13.437750 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:13.437852 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:13.438072 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:52:13.438078 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:52:13.438199 (+ 121us) client_negotiation.cc:769] Sending connection context
0808 16:52:13.438281 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 16:52:13.438454 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":74,"threads_started":1}
I20260808 16:52:13.439018 12645 heartbeater.cc:347] Connected to a master server at 127.8.24.62:43237
I20260808 16:52:13.439100 12645 heartbeater.cc:464] Registering TS with master...
I20260808 16:52:13.439253 12645 heartbeater.cc:511] Master 127.8.24.62:43237 requested a full tablet report, sending...
I20260808 16:52:13.439570 11943 ts_manager.cc:194] Registered new tserver with Master: af049643d76c4b96b39eb51de88e6270 (127.8.24.3:36957)
I20260808 16:52:13.439910 11943 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.3@KRBTEST.COM'} at 127.8.24.3:43297
I20260808 16:52:13.445724 8288 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 16:52:13.457885 12007 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:52:13.456909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50462 (local address 127.8.24.62:43237)
0808 16:52:13.456947 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:13.456949 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:13.457069 (+ 120us) server_negotiation.cc:407] Connection header received
0808 16:52:13.457151 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:13.457152 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:13.457161 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:13.457227 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:13.457810 (+ 583us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50462: BlockingRecv error: recv got EOF from 127.0.0.1:50462 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 16:52:13.458928 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 12035
I20260808 16:52:13.470821 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 12237
I20260808 16:52:13.481451 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 12452
I20260808 16:52:13.497670 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 11894
2026-08-08T16:52:13Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (1915 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786207919384338-8288-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 16:52:13 dist-test-slave-6zbq krb5kdc[12664](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 16:52:13 dist-test-slave-6zbq krb5kdc[12664](info): set up 2 sockets
Aug 08 16:52:13 dist-test-slave-6zbq krb5kdc[12664](info): commencing operation
krb5kdc: starting...
W20260808 16:52:14.638121 8288 mini_kdc.cc:121] Time spent starting KDC: real 1.113s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786207919384338-8288-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786207919384338-8288-0/krb5kdc/test-user.keytab.
Aug 08 16:52:14 dist-test-slave-6zbq krb5kdc[12664](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207934, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T16:52:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T16:52:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.24.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
I20260808 16:52:14.744303 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786207919384338-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786207919384338-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786207919384338-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:34783
--webserver_interface=127.8.24.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:37249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:34783
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:52:14.822628 12688 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:52:14.822815 12688 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:52:14.822844 12688 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:52:14.823982 12688 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 16:52:14.824023 12688 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:52:14.824041 12688 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:52:14.824056 12688 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 16:52:14.824074 12688 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 16:52:14.825644 12688 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:37249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786207919384338-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786207919384338-8288-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:34783
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:34783
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.24.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12688
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786207919384338-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:52:14.825872 12688 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:52:14.826064 12688 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 16:52:14.830536 12688 server_base.cc:1034] running on GCE node
W20260808 16:52:14.830509 12696 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:14.830518 12697 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:14.830528 12701 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:14.830948 12688 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:52:14.831207 12688 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:52:14.832333 12688 hybrid_clock.cc:648] HybridClock initialized: now 1786207934832329 us; error 32 us; skew 500 ppm
Aug 08 16:52:14 dist-test-slave-6zbq krb5kdc[12664](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207934, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:52:14.834546 12688 init.cc:377] Logged in from keytab as kudu/127.8.24.62@KRBTEST.COM (short username kudu)
I20260808 16:52:14.834957 12688 webserver.cc:460] Webserver started at http://127.8.24.62:34491/ using document root <none> and password file <none>
I20260808 16:52:14.835141 12688 fs_manager.cc:359] Metadata directory not provided
I20260808 16:52:14.835194 12688 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:52:14.835314 12688 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:52:14.836277 12688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786207919384338-8288-0/minicluster-data/master-0/data/instance:
uuid: "6439a65c1a7145cda3ded5b3adf41c6c"
format_stamp: "Formatted at 2026-08-08 16:52:14 on dist-test-slave-6zbq"
server_key: "8b1dbe8de292e212c99c87f584c37a28"
server_key_iv: "fab94c60864da823258e736b2d885402"
server_key_version: "encryptionkey@0"
I20260808 16:52:14.836596 12688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786207919384338-8288-0/minicluster-data/master-0/wal/instance:
uuid: "6439a65c1a7145cda3ded5b3adf41c6c"
format_stamp: "Formatted at 2026-08-08 16:52:14 on dist-test-slave-6zbq"
server_key: "8b1dbe8de292e212c99c87f584c37a28"
server_key_iv: "fab94c60864da823258e736b2d885402"
server_key_version: "encryptionkey@0"
I20260808 16:52:14.837953 12688 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 16:52:14.838678 12711 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:14.838838 12688 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:14.838922 12688 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786207919384338-8288-0/minicluster-data/master-0/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786207919384338-8288-0/minicluster-data/master-0/wal
uuid: "6439a65c1a7145cda3ded5b3adf41c6c"
format_stamp: "Formatted at 2026-08-08 16:52:14 on dist-test-slave-6zbq"
server_key: "8b1dbe8de292e212c99c87f584c37a28"
server_key_iv: "fab94c60864da823258e736b2d885402"
server_key_version: "encryptionkey@0"
I20260808 16:52:14.838980 12688 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786207919384338-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786207919384338-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786207919384338-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:52:14.851493 12688 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:52:14.853327 12688 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:52:14.853452 12688 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:52:14.857149 12688 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.62:34783
I20260808 16:52:14.857178 12787 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.62:34783 every 8 connection(s)
I20260808 16:52:14.857467 12688 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
I20260808 16:52:14.858172 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 12688
I20260808 16:52:14.858249 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786207919384338-8288-0/minicluster-data/master-0/wal/instance
I20260808 16:52:14.858306 12788 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:52:14.858446 8288 external_mini_cluster.cc:1428] Setting key a13794a7c8b8c838e3b6addfaee95002
I20260808 16:52:14.861966 12788 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6439a65c1a7145cda3ded5b3adf41c6c: Bootstrap starting.
I20260808 16:52:14.862763 12788 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6439a65c1a7145cda3ded5b3adf41c6c: Neither blocks nor log segments found. Creating new log.
I20260808 16:52:14.863090 12788 log.cc:824] T 00000000000000000000000000000000 P 6439a65c1a7145cda3ded5b3adf41c6c: Log is configured to *not* fsync() on all Append() calls
I20260808 16:52:14.863894 12788 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6439a65c1a7145cda3ded5b3adf41c6c: No bootstrap required, opened a new log
Aug 08 16:52:14 dist-test-slave-6zbq krb5kdc[12664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207934, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260808 16:52:14.865777 12788 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6439a65c1a7145cda3ded5b3adf41c6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6439a65c1a7145cda3ded5b3adf41c6c" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 34783 } }
I20260808 16:52:14.865936 12788 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6439a65c1a7145cda3ded5b3adf41c6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:52:14.865969 12788 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6439a65c1a7145cda3ded5b3adf41c6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6439a65c1a7145cda3ded5b3adf41c6c, State: Initialized, Role: FOLLOWER
I20260808 16:52:14.866112 12788 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6439a65c1a7145cda3ded5b3adf41c6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6439a65c1a7145cda3ded5b3adf41c6c" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 34783 } }
I20260808 16:52:14.866187 12788 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6439a65c1a7145cda3ded5b3adf41c6c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 16:52:14.866237 12788 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6439a65c1a7145cda3ded5b3adf41c6c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 16:52:14.866333 12788 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6439a65c1a7145cda3ded5b3adf41c6c [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:52:14.866925 12788 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6439a65c1a7145cda3ded5b3adf41c6c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6439a65c1a7145cda3ded5b3adf41c6c" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 34783 } }
I20260808 16:52:14.867095 12788 leader_election.cc:302] T 00000000000000000000000000000000 P 6439a65c1a7145cda3ded5b3adf41c6c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6439a65c1a7145cda3ded5b3adf41c6c; no voters:
I20260808 16:52:14.867256 12788 leader_election.cc:288] T 00000000000000000000000000000000 P 6439a65c1a7145cda3ded5b3adf41c6c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 16:52:14.867303 12797 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6439a65c1a7145cda3ded5b3adf41c6c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:52:14.867497 12797 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6439a65c1a7145cda3ded5b3adf41c6c [term 1 LEADER]: Becoming Leader. State: Replica: 6439a65c1a7145cda3ded5b3adf41c6c, State: Running, Role: LEADER
I20260808 16:52:14.867578 12788 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6439a65c1a7145cda3ded5b3adf41c6c [sys.catalog]: configured and running, proceeding with master startup.
I20260808 16:52:14.867662 12797 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6439a65c1a7145cda3ded5b3adf41c6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6439a65c1a7145cda3ded5b3adf41c6c" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 34783 } }
I20260808 16:52:14.868041 12797 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6439a65c1a7145cda3ded5b3adf41c6c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6439a65c1a7145cda3ded5b3adf41c6c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6439a65c1a7145cda3ded5b3adf41c6c" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 34783 } } }
I20260808 16:52:14.868193 12797 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6439a65c1a7145cda3ded5b3adf41c6c [sys.catalog]: This master's current role is: LEADER
I20260808 16:52:14.868387 12797 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6439a65c1a7145cda3ded5b3adf41c6c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6439a65c1a7145cda3ded5b3adf41c6c. Latest consensus state: current_term: 1 leader_uuid: "6439a65c1a7145cda3ded5b3adf41c6c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6439a65c1a7145cda3ded5b3adf41c6c" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 34783 } } }
I20260808 16:52:14.868526 12797 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6439a65c1a7145cda3ded5b3adf41c6c [sys.catalog]: This master's current role is: LEADER
I20260808 16:52:14.868738 12812 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 16:52:14.868942 12795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:14.858995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39250 (local address 127.8.24.62:34783)
0808 16:52:14.859139 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:14.859147 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:14.859168 (+ 21us) server_negotiation.cc:407] Connection header received
0808 16:52:14.859540 (+ 372us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:14.859546 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:14.859589 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:14.859695 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:14.860451 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:14.861319 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:14.862056 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:14.862510 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:14.862603 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:14.864881 (+ 2278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:14.864885 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:14.864885 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:14.864897 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:14.866933 (+ 2036us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:14.867289 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:14.867292 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:14.867293 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:14.867450 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:14.867666 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:14.867667 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:14.867667 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:14.867976 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:14.868100 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:14.868604 (+ 504us) server_negotiation.cc:299] Negotiation successful
0808 16:52:14.868739 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":70,"threads_started":1}
I20260808 16:52:14.869334 12812 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 16:52:14.869997 8288 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 16:52:14.871352 12812 catalog_manager.cc:1347] Generated new cluster ID: f635e71ca0ab41a29c457ef0bf89438d
I20260808 16:52:14.871405 12812 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 16:52:14.890383 12812 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 16:52:14.890908 12812 catalog_manager.cc:1504] Loading token signing keys...
I20260808 16:52:14.890960 12795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:14.886962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39260 (local address 127.8.24.62:34783)
0808 16:52:14.887015 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:14.887017 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:14.887155 (+ 138us) server_negotiation.cc:407] Connection header received
0808 16:52:14.887200 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:14.887201 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:14.887210 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:14.887261 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:14.887633 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:14.888364 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:14.889358 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:14.889664 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:14.889727 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:14.890451 (+ 724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:14.890453 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:14.890453 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 16:52:14.890644 (+ 191us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:14.890661 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 16:52:14.890667 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:14.890698 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:14.890835 (+ 137us) server_negotiation.cc:299] Negotiation successful
0808 16:52:14.890856 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 16:52:14.894809 12812 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6439a65c1a7145cda3ded5b3adf41c6c: Generated new TSK 0
I20260808 16:52:14.894960 12812 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 16:52:14.897804 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 12688
2026-08-08T16:52:14Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1400 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786207919384338-8288-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 16:52:15 dist-test-slave-6zbq krb5kdc[12836](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 16:52:15 dist-test-slave-6zbq krb5kdc[12836](info): set up 2 sockets
Aug 08 16:52:15 dist-test-slave-6zbq krb5kdc[12836](info): commencing operation
krb5kdc: starting...
W20260808 16:52:16.146662 8288 mini_kdc.cc:121] Time spent starting KDC: real 1.221s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786207919384338-8288-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786207919384338-8288-0/krb5kdc/test-user.keytab.
Aug 08 16:52:16 dist-test-slave-6zbq krb5kdc[12836](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207936, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T16:52:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T16:52:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.24.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
I20260808 16:52:16.252144 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786207919384338-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786207919384338-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786207919384338-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:33683
--webserver_interface=127.8.24.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:34609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:33683
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:52:16.330505 12858 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:52:16.330687 12858 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:52:16.330708 12858 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:52:16.331792 12858 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 16:52:16.331832 12858 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:52:16.331846 12858 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:52:16.331856 12858 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 16:52:16.331867 12858 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 16:52:16.333262 12858 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:34609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786207919384338-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786207919384338-8288-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:33683
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:33683
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.24.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12858
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786207919384338-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:52:16.333473 12858 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:52:16.333782 12858 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:52:16.338163 12872 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:16.338151 12868 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:16.338155 12867 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:16.338213 12858 server_base.cc:1034] running on GCE node
I20260808 16:52:16.338554 12858 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:52:16.338807 12858 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:52:16.339946 12858 hybrid_clock.cc:648] HybridClock initialized: now 1786207936339932 us; error 27 us; skew 500 ppm
Aug 08 16:52:16 dist-test-slave-6zbq krb5kdc[12836](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207936, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:52:16.342103 12858 init.cc:377] Logged in from keytab as kudu/127.8.24.62@KRBTEST.COM (short username kudu)
I20260808 16:52:16.342514 12858 webserver.cc:460] Webserver started at http://127.8.24.62:33889/ using document root <none> and password file <none>
I20260808 16:52:16.342698 12858 fs_manager.cc:359] Metadata directory not provided
I20260808 16:52:16.342743 12858 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:52:16.342862 12858 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:52:16.343801 12858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786207919384338-8288-0/minicluster-data/master-0/data/instance:
uuid: "d6532c68ca9648878e1b635633ba7e3d"
format_stamp: "Formatted at 2026-08-08 16:52:16 on dist-test-slave-6zbq"
server_key: "f61a7430c290a6c09d3d299a9a7f356a"
server_key_iv: "64ab3a7eeefb95d011048c08dd70ce35"
server_key_version: "encryptionkey@0"
I20260808 16:52:16.344113 12858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786207919384338-8288-0/minicluster-data/master-0/wal/instance:
uuid: "d6532c68ca9648878e1b635633ba7e3d"
format_stamp: "Formatted at 2026-08-08 16:52:16 on dist-test-slave-6zbq"
server_key: "f61a7430c290a6c09d3d299a9a7f356a"
server_key_iv: "64ab3a7eeefb95d011048c08dd70ce35"
server_key_version: "encryptionkey@0"
I20260808 16:52:16.345271 12858 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 16:52:16.345894 12881 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:16.346053 12858 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 16:52:16.346132 12858 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786207919384338-8288-0/minicluster-data/master-0/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786207919384338-8288-0/minicluster-data/master-0/wal
uuid: "d6532c68ca9648878e1b635633ba7e3d"
format_stamp: "Formatted at 2026-08-08 16:52:16 on dist-test-slave-6zbq"
server_key: "f61a7430c290a6c09d3d299a9a7f356a"
server_key_iv: "64ab3a7eeefb95d011048c08dd70ce35"
server_key_version: "encryptionkey@0"
I20260808 16:52:16.346204 12858 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786207919384338-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786207919384338-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786207919384338-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:52:16.363722 12858 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:52:16.365232 12858 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:52:16.365346 12858 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:52:16.368855 12858 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.62:33683
I20260808 16:52:16.368889 12962 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.62:33683 every 8 connection(s)
I20260808 16:52:16.369156 12858 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
I20260808 16:52:16.370209 12964 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:52:16.372233 12964 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6532c68ca9648878e1b635633ba7e3d: Bootstrap starting.
I20260808 16:52:16.372726 12964 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d6532c68ca9648878e1b635633ba7e3d: Neither blocks nor log segments found. Creating new log.
I20260808 16:52:16.372946 12964 log.cc:824] T 00000000000000000000000000000000 P d6532c68ca9648878e1b635633ba7e3d: Log is configured to *not* fsync() on all Append() calls
I20260808 16:52:16.373545 12964 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6532c68ca9648878e1b635633ba7e3d: No bootstrap required, opened a new log
I20260808 16:52:16.375090 12964 raft_consensus.cc:348] T 00000000000000000000000000000000 P d6532c68ca9648878e1b635633ba7e3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6532c68ca9648878e1b635633ba7e3d" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 33683 } }
I20260808 16:52:16.375217 12964 raft_consensus.cc:374] T 00000000000000000000000000000000 P d6532c68ca9648878e1b635633ba7e3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:52:16.375244 12964 raft_consensus.cc:729] T 00000000000000000000000000000000 P d6532c68ca9648878e1b635633ba7e3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6532c68ca9648878e1b635633ba7e3d, State: Initialized, Role: FOLLOWER
I20260808 16:52:16.375335 12964 consensus_queue.cc:260] T 00000000000000000000000000000000 P d6532c68ca9648878e1b635633ba7e3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6532c68ca9648878e1b635633ba7e3d" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 33683 } }
I20260808 16:52:16.375382 12964 raft_consensus.cc:388] T 00000000000000000000000000000000 P d6532c68ca9648878e1b635633ba7e3d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 16:52:16.375401 12964 raft_consensus.cc:482] T 00000000000000000000000000000000 P d6532c68ca9648878e1b635633ba7e3d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 16:52:16.375428 12964 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d6532c68ca9648878e1b635633ba7e3d [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:52:16.376111 12964 raft_consensus.cc:504] T 00000000000000000000000000000000 P d6532c68ca9648878e1b635633ba7e3d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6532c68ca9648878e1b635633ba7e3d" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 33683 } }
I20260808 16:52:16.376178 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 12858
I20260808 16:52:16.376199 12964 leader_election.cc:302] T 00000000000000000000000000000000 P d6532c68ca9648878e1b635633ba7e3d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d6532c68ca9648878e1b635633ba7e3d; no voters:
I20260808 16:52:16.376273 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786207919384338-8288-0/minicluster-data/master-0/wal/instance
I20260808 16:52:16.376339 12964 leader_election.cc:288] T 00000000000000000000000000000000 P d6532c68ca9648878e1b635633ba7e3d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 16:52:16.376399 12967 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d6532c68ca9648878e1b635633ba7e3d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:52:16.376446 8288 external_mini_cluster.cc:1428] Setting key dc305e1ae8ba8ceab71703b0b0551f40
I20260808 16:52:16.376516 12967 raft_consensus.cc:686] T 00000000000000000000000000000000 P d6532c68ca9648878e1b635633ba7e3d [term 1 LEADER]: Becoming Leader. State: Replica: d6532c68ca9648878e1b635633ba7e3d, State: Running, Role: LEADER
I20260808 16:52:16.376613 12964 sys_catalog.cc:564] T 00000000000000000000000000000000 P d6532c68ca9648878e1b635633ba7e3d [sys.catalog]: configured and running, proceeding with master startup.
I20260808 16:52:16.376595 12967 consensus_queue.cc:237] T 00000000000000000000000000000000 P d6532c68ca9648878e1b635633ba7e3d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6532c68ca9648878e1b635633ba7e3d" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 33683 } }
I20260808 16:52:16.377049 12969 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6532c68ca9648878e1b635633ba7e3d [sys.catalog]: SysCatalogTable state changed. Reason: New leader d6532c68ca9648878e1b635633ba7e3d. Latest consensus state: current_term: 1 leader_uuid: "d6532c68ca9648878e1b635633ba7e3d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6532c68ca9648878e1b635633ba7e3d" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 33683 } } }
I20260808 16:52:16.377060 12968 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6532c68ca9648878e1b635633ba7e3d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d6532c68ca9648878e1b635633ba7e3d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6532c68ca9648878e1b635633ba7e3d" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 33683 } } }
I20260808 16:52:16.377213 12969 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6532c68ca9648878e1b635633ba7e3d [sys.catalog]: This master's current role is: LEADER
I20260808 16:52:16.377466 12968 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6532c68ca9648878e1b635633ba7e3d [sys.catalog]: This master's current role is: LEADER
I20260808 16:52:16.377636 12981 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 16:52:16.378140 12981 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 16:52:16.379911 12981 catalog_manager.cc:1347] Generated new cluster ID: 6b2b1c5440ad492caecf664a3fcde13d
I20260808 16:52:16.379963 12981 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 16:52:16 dist-test-slave-6zbq krb5kdc[12836](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207936, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260808 16:52:16.388109 12975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:16.376827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42384 (local address 127.8.24.62:33683)
0808 16:52:16.377332 (+ 505us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:16.377341 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:16.377375 (+ 34us) server_negotiation.cc:407] Connection header received
0808 16:52:16.378175 (+ 800us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:16.378182 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:16.378242 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:16.378956 (+ 714us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:16.379181 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:16.380502 (+ 1321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:16.381528 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:16.382143 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:16.382254 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:16.384814 (+ 2560us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:16.384818 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:16.384818 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:16.384829 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:16.386622 (+ 1793us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:16.386995 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:16.386998 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:16.386998 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:16.387091 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:16.387267 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:16.387267 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:16.387268 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:16.387537 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:16.387689 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:16.387776 (+ 87us) server_negotiation.cc:299] Negotiation successful
0808 16:52:16.387899 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":461,"thread_start_us":71,"threads_started":1}
I20260808 16:52:16.392262 12981 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 16:52:16.392706 12981 catalog_manager.cc:1504] Loading token signing keys...
I20260808 16:52:16.397122 12981 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d6532c68ca9648878e1b635633ba7e3d: Generated new TSK 0
I20260808 16:52:16.397274 12981 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.24.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.1.keytab.
Entry for principal kudu/127.8.24.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.1.keytab.
I20260808 16:52:16.423274 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786207919384338-8288-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786207919384338-8288-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786207919384338-8288-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786207919384338-8288-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.1:0
--local_ip_for_outbound_sockets=127.8.24.1
--webserver_interface=127.8.24.1
--webserver_port=0
--tserver_master_addrs=127.8.24.62:33683
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.1.keytab
--principal=kudu/127.8.24.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:34609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:52:16.502378 13000 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:52:16.502555 13000 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:52:16.502578 13000 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:52:16.503650 13000 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:52:16.503683 13000 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:52:16.503706 13000 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.1
I20260808 16:52:16.505041 13000 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:34609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786207919384338-8288-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786207919384338-8288-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.1.keytab
--principal=kudu/127.8.24.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786207919384338-8288-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.24.1
--webserver_port=0
--tserver_master_addrs=127.8.24.62:33683
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13000
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.1
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786207919384338-8288-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:52:16.505223 13000 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:52:16.505395 13000 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:52:16.509915 13008 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:16.509891 13009 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:16.510030 13000 server_base.cc:1034] running on GCE node
W20260808 16:52:16.509910 13012 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:16.510315 13000 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:52:16.510587 13000 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:52:16.511723 13000 hybrid_clock.cc:648] HybridClock initialized: now 1786207936511711 us; error 28 us; skew 500 ppm
Aug 08 16:52:16 dist-test-slave-6zbq krb5kdc[12836](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207936, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:52:16.514088 13000 init.cc:377] Logged in from keytab as kudu/127.8.24.1@KRBTEST.COM (short username kudu)
I20260808 16:52:16.514532 13000 webserver.cc:460] Webserver started at http://127.8.24.1:44787/ using document root <none> and password file <none>
I20260808 16:52:16.514757 13000 fs_manager.cc:359] Metadata directory not provided
I20260808 16:52:16.514803 13000 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:52:16.514925 13000 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:52:16.515867 13000 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786207919384338-8288-0/minicluster-data/ts-0/data/instance:
uuid: "ad0ae83144c04dbfb96575de8c507e00"
format_stamp: "Formatted at 2026-08-08 16:52:16 on dist-test-slave-6zbq"
server_key: "d8df08d96f3f4e8c1c071b0c81f3a909"
server_key_iv: "e317b954a187ac174571894b5093691c"
server_key_version: "encryptionkey@0"
I20260808 16:52:16.516180 13000 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786207919384338-8288-0/minicluster-data/ts-0/wal/instance:
uuid: "ad0ae83144c04dbfb96575de8c507e00"
format_stamp: "Formatted at 2026-08-08 16:52:16 on dist-test-slave-6zbq"
server_key: "d8df08d96f3f4e8c1c071b0c81f3a909"
server_key_iv: "e317b954a187ac174571894b5093691c"
server_key_version: "encryptionkey@0"
I20260808 16:52:16.517400 13000 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 16:52:16.518033 13021 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:16.518183 13000 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 16:52:16.518265 13000 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786207919384338-8288-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786207919384338-8288-0/minicluster-data/ts-0/wal
uuid: "ad0ae83144c04dbfb96575de8c507e00"
format_stamp: "Formatted at 2026-08-08 16:52:16 on dist-test-slave-6zbq"
server_key: "d8df08d96f3f4e8c1c071b0c81f3a909"
server_key_iv: "e317b954a187ac174571894b5093691c"
server_key_version: "encryptionkey@0"
I20260808 16:52:16.518330 13000 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786207919384338-8288-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786207919384338-8288-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786207919384338-8288-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:52:16.530179 13000 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:52:16.532125 13000 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:52:16.532258 13000 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:52:16.532442 13000 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:52:16.532703 13000 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:52:16.532735 13000 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:16.532759 13000 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:52:16.532786 13000 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:16.538024 13000 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.1:37153
I20260808 16:52:16.538079 13196 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.1:37153 every 8 connection(s)
I20260808 16:52:16.538357 13000 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786207919384338-8288-0/minicluster-data/ts-0/data/info.pb
Aug 08 16:52:16 dist-test-slave-6zbq krb5kdc[12836](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207936, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.1@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260808 16:52:16.546415 12975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:16.539001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:45281 (local address 127.8.24.62:33683)
0808 16:52:16.539065 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:16.539067 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:16.539228 (+ 161us) server_negotiation.cc:407] Connection header received
0808 16:52:16.539688 (+ 460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:16.539689 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:16.539703 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:16.539753 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:16.540604 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:16.541113 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:16.542059 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:16.542271 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:16.542328 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:16.544155 (+ 1827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:16.544157 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:16.544157 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:16.544165 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:16.545191 (+ 1026us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:16.545542 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:16.545543 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:16.545543 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:16.545597 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:16.545806 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:16.545807 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:16.545807 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:16.545915 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:16.545970 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:16.546216 (+ 246us) server_negotiation.cc:299] Negotiation successful
0808 16:52:16.546304 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 16:52:16.546550 13201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:16.538960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:33683 (local address 127.8.24.1:45281)
0808 16:52:16.539114 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:16.539152 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:16.539591 (+ 439us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:16.539818 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:16.539822 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:16.540336 (+ 514us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:16.540505 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:16.540510 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:16.541197 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:16.541198 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:16.541969 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:16.541971 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:16.542358 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:16.542359 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:16.542475 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:16.542845 (+ 370us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:16.542852 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:16.544069 (+ 1217us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:52:16.545264 (+ 1195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:16.545270 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:16.545280 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:16.545474 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:16.545650 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:16.545651 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:16.545651 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:16.545745 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:16.545970 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:52:16.545977 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:52:16.546099 (+ 122us) client_negotiation.cc:769] Sending connection context
0808 16:52:16.546181 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 16:52:16.546327 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":77,"threads_started":1}
I20260808 16:52:16.546967 13197 heartbeater.cc:347] Connected to a master server at 127.8.24.62:33683
I20260808 16:52:16.547049 13197 heartbeater.cc:464] Registering TS with master...
I20260808 16:52:16.547119 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 13000
I20260808 16:52:16.547201 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786207919384338-8288-0/minicluster-data/ts-0/wal/instance
I20260808 16:52:16.547220 13197 heartbeater.cc:511] Master 127.8.24.62:33683 requested a full tablet report, sending...
I20260808 16:52:16.547379 8288 external_mini_cluster.cc:1428] Setting key f2f522f3451564a6362d3126abd98323
I20260808 16:52:16.547775 12907 ts_manager.cc:194] Registered new tserver with Master: ad0ae83144c04dbfb96575de8c507e00 (127.8.24.1:37153)
I20260808 16:52:16.548539 12907 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.1@KRBTEST.COM'} at 127.8.24.1:45281
WARNING: no policy specified for kudu/127.8.24.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.2.keytab.
Entry for principal kudu/127.8.24.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.2.keytab.
I20260808 16:52:16.568353 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786207919384338-8288-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786207919384338-8288-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786207919384338-8288-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786207919384338-8288-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.2:0
--local_ip_for_outbound_sockets=127.8.24.2
--webserver_interface=127.8.24.2
--webserver_port=0
--tserver_master_addrs=127.8.24.62:33683
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.2.keytab
--principal=kudu/127.8.24.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:34609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:52:16.647305 13205 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:52:16.647483 13205 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:52:16.647505 13205 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:52:16.648613 13205 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:52:16.648653 13205 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:52:16.648682 13205 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.2
I20260808 16:52:16.650136 13205 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:34609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786207919384338-8288-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786207919384338-8288-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.2.keytab
--principal=kudu/127.8.24.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786207919384338-8288-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.24.2
--webserver_port=0
--tserver_master_addrs=127.8.24.62:33683
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13205
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.2
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786207919384338-8288-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:52:16.650307 13205 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:52:16.650493 13205 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 16:52:16.654978 13205 server_base.cc:1034] running on GCE node
W20260808 16:52:16.654912 13214 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:16.654904 13213 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:16.654903 13217 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:16.655297 13205 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:52:16.655539 13205 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:52:16.656684 13205 hybrid_clock.cc:648] HybridClock initialized: now 1786207936656670 us; error 27 us; skew 500 ppm
Aug 08 16:52:16 dist-test-slave-6zbq krb5kdc[12836](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207936, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:52:16.664024 13205 init.cc:377] Logged in from keytab as kudu/127.8.24.2@KRBTEST.COM (short username kudu)
I20260808 16:52:16.664418 13205 webserver.cc:460] Webserver started at http://127.8.24.2:33377/ using document root <none> and password file <none>
I20260808 16:52:16.664602 13205 fs_manager.cc:359] Metadata directory not provided
I20260808 16:52:16.664659 13205 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:52:16.664778 13205 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:52:16.665707 13205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786207919384338-8288-0/minicluster-data/ts-1/data/instance:
uuid: "a02ed8732bf74e7c9ed78008694707fb"
format_stamp: "Formatted at 2026-08-08 16:52:16 on dist-test-slave-6zbq"
server_key: "475a421300b98c1faa39c4d80db5674a"
server_key_iv: "893ef479fcf87d973a7275e3f4c85066"
server_key_version: "encryptionkey@0"
I20260808 16:52:16.666020 13205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786207919384338-8288-0/minicluster-data/ts-1/wal/instance:
uuid: "a02ed8732bf74e7c9ed78008694707fb"
format_stamp: "Formatted at 2026-08-08 16:52:16 on dist-test-slave-6zbq"
server_key: "475a421300b98c1faa39c4d80db5674a"
server_key_iv: "893ef479fcf87d973a7275e3f4c85066"
server_key_version: "encryptionkey@0"
I20260808 16:52:16.667281 13205 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 16:52:16.667902 13232 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:16.668049 13205 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 16:52:16.668130 13205 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786207919384338-8288-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786207919384338-8288-0/minicluster-data/ts-1/wal
uuid: "a02ed8732bf74e7c9ed78008694707fb"
format_stamp: "Formatted at 2026-08-08 16:52:16 on dist-test-slave-6zbq"
server_key: "475a421300b98c1faa39c4d80db5674a"
server_key_iv: "893ef479fcf87d973a7275e3f4c85066"
server_key_version: "encryptionkey@0"
I20260808 16:52:16.668190 13205 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786207919384338-8288-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786207919384338-8288-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786207919384338-8288-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:52:16.682442 13205 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:52:16.684351 13205 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:52:16.684479 13205 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:52:16.684674 13205 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:52:16.684949 13205 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:52:16.684983 13205 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:16.685007 13205 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:52:16.685025 13205 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:16.690213 13205 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.2:39415
I20260808 16:52:16.690248 13403 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.2:39415 every 8 connection(s)
I20260808 16:52:16.690534 13205 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786207919384338-8288-0/minicluster-data/ts-1/data/info.pb
I20260808 16:52:16.692301 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 13205
I20260808 16:52:16.692404 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786207919384338-8288-0/minicluster-data/ts-1/wal/instance
I20260808 16:52:16.692595 8288 external_mini_cluster.cc:1428] Setting key 6d7068392a93a6358013eef2279f4d60
Aug 08 16:52:16 dist-test-slave-6zbq krb5kdc[12836](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207936, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.2@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260808 16:52:16.699640 12975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:16.691165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:42023 (local address 127.8.24.62:33683)
0808 16:52:16.691220 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:16.691222 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:16.691375 (+ 153us) server_negotiation.cc:407] Connection header received
0808 16:52:16.691847 (+ 472us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:16.691849 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:16.691862 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:16.691913 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:16.692974 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:16.693587 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:16.694585 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:16.694861 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:16.694926 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:16.697127 (+ 2201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:16.697129 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:16.697129 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:16.697136 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:16.698406 (+ 1270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:16.698838 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:16.698839 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:16.698839 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:16.698885 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:16.699105 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:16.699105 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:16.699106 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:16.699193 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:16.699237 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:16.699522 (+ 285us) server_negotiation.cc:299] Negotiation successful
0808 16:52:16.699580 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 16:52:16.699942 13407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:16.691131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:33683 (local address 127.8.24.2:42023)
0808 16:52:16.691272 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:16.691304 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:16.691752 (+ 448us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:16.691980 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:16.691983 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:16.692664 (+ 681us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:16.692884 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:16.692889 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:16.693676 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:16.693677 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:16.694509 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:16.694511 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:16.694958 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:16.694960 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:16.695076 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:16.695551 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:16.695559 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:16.697033 (+ 1474us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:52:16.698466 (+ 1433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:16.698473 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:16.698475 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:16.698768 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:16.698926 (+ 158us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:16.698926 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:16.698927 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:16.699050 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:16.699249 (+ 199us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:52:16.699255 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:52:16.699404 (+ 149us) client_negotiation.cc:769] Sending connection context
0808 16:52:16.699514 (+ 110us) client_negotiation.cc:240] Negotiation successful
0808 16:52:16.699677 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":75,"threads_started":1}
I20260808 16:52:16.700415 13404 heartbeater.cc:347] Connected to a master server at 127.8.24.62:33683
I20260808 16:52:16.700498 13404 heartbeater.cc:464] Registering TS with master...
I20260808 16:52:16.700657 13404 heartbeater.cc:511] Master 127.8.24.62:33683 requested a full tablet report, sending...
I20260808 16:52:16.701028 12907 ts_manager.cc:194] Registered new tserver with Master: a02ed8732bf74e7c9ed78008694707fb (127.8.24.2:39415)
WARNING: no policy specified for kudu/127.8.24.3@KRBTEST.COM; defaulting to no policy
I20260808 16:52:16.701419 12907 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.2@KRBTEST.COM'} at 127.8.24.2:42023
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.3.keytab.
Entry for principal kudu/127.8.24.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.3.keytab.
I20260808 16:52:16.718297 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786207919384338-8288-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786207919384338-8288-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786207919384338-8288-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786207919384338-8288-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.3:0
--local_ip_for_outbound_sockets=127.8.24.3
--webserver_interface=127.8.24.3
--webserver_port=0
--tserver_master_addrs=127.8.24.62:33683
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.3.keytab
--principal=kudu/127.8.24.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:34609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:52:16.795862 13411 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:52:16.796038 13411 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:52:16.796067 13411 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:52:16.797171 13411 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:52:16.797210 13411 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:52:16.797243 13411 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.3
I20260808 16:52:16.798623 13411 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:34609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786207919384338-8288-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786207919384338-8288-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.3.keytab
--principal=kudu/127.8.24.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786207919384338-8288-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.24.3
--webserver_port=0
--tserver_master_addrs=127.8.24.62:33683
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13411
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.3
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786207919384338-8288-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:52:16.798846 13411 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:52:16.799043 13411 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:52:16.803647 13426 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:16.803654 13422 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:16.803654 13423 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:16.803725 13411 server_base.cc:1034] running on GCE node
I20260808 16:52:16.803968 13411 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:52:16.804191 13411 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:52:16.805330 13411 hybrid_clock.cc:648] HybridClock initialized: now 1786207936805313 us; error 29 us; skew 500 ppm
Aug 08 16:52:16 dist-test-slave-6zbq krb5kdc[12836](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207936, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:52:16.807538 13411 init.cc:377] Logged in from keytab as kudu/127.8.24.3@KRBTEST.COM (short username kudu)
I20260808 16:52:16.807926 13411 webserver.cc:460] Webserver started at http://127.8.24.3:37381/ using document root <none> and password file <none>
I20260808 16:52:16.808139 13411 fs_manager.cc:359] Metadata directory not provided
I20260808 16:52:16.808185 13411 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:52:16.808305 13411 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:52:16.809252 13411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786207919384338-8288-0/minicluster-data/ts-2/data/instance:
uuid: "4f67afeb23314b9bb17575080194a91d"
format_stamp: "Formatted at 2026-08-08 16:52:16 on dist-test-slave-6zbq"
server_key: "2ae1922d8248c23e23073732228b2808"
server_key_iv: "d3f704db7cf2106e86f72bd74cfd35d0"
server_key_version: "encryptionkey@0"
I20260808 16:52:16.809558 13411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786207919384338-8288-0/minicluster-data/ts-2/wal/instance:
uuid: "4f67afeb23314b9bb17575080194a91d"
format_stamp: "Formatted at 2026-08-08 16:52:16 on dist-test-slave-6zbq"
server_key: "2ae1922d8248c23e23073732228b2808"
server_key_iv: "d3f704db7cf2106e86f72bd74cfd35d0"
server_key_version: "encryptionkey@0"
I20260808 16:52:16.810779 13411 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260808 16:52:16.811420 13437 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:16.811565 13411 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 16:52:16.811646 13411 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786207919384338-8288-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786207919384338-8288-0/minicluster-data/ts-2/wal
uuid: "4f67afeb23314b9bb17575080194a91d"
format_stamp: "Formatted at 2026-08-08 16:52:16 on dist-test-slave-6zbq"
server_key: "2ae1922d8248c23e23073732228b2808"
server_key_iv: "d3f704db7cf2106e86f72bd74cfd35d0"
server_key_version: "encryptionkey@0"
I20260808 16:52:16.811704 13411 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786207919384338-8288-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786207919384338-8288-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786207919384338-8288-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:52:16.817895 13411 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:52:16.819481 13411 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:52:16.819607 13411 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:52:16.819782 13411 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:52:16.820025 13411 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:52:16.820057 13411 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:16.820083 13411 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:52:16.820117 13411 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:16.825452 13411 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.3:45419
I20260808 16:52:16.825516 13614 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.3:45419 every 8 connection(s)
I20260808 16:52:16.825748 13411 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786207919384338-8288-0/minicluster-data/ts-2/data/info.pb
Aug 08 16:52:16 dist-test-slave-6zbq krb5kdc[12836](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207936, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.3@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260808 16:52:16.832320 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 13411
I20260808 16:52:16.832412 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786207919384338-8288-0/minicluster-data/ts-2/wal/instance
I20260808 16:52:16.832592 8288 external_mini_cluster.cc:1428] Setting key 00cbb807a862e814092d1d1808a10222
I20260808 16:52:16.833750 12975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:16.826317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:57111 (local address 127.8.24.62:33683)
0808 16:52:16.826381 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:16.826383 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:16.826576 (+ 193us) server_negotiation.cc:407] Connection header received
0808 16:52:16.826987 (+ 411us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:16.826989 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:16.827001 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:16.827045 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:16.827832 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:16.828375 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:16.829310 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:16.829499 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:16.829552 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:16.831376 (+ 1824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:16.831377 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:16.831378 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:16.831384 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:16.832466 (+ 1082us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:16.832875 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:16.832876 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:16.832876 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:16.832937 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:16.833195 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:16.833196 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:16.833196 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:16.833312 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:16.833369 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:16.833628 (+ 259us) server_negotiation.cc:299] Negotiation successful
0808 16:52:16.833689 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 16:52:16.833871 13618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:16.826288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:33683 (local address 127.8.24.3:57111)
0808 16:52:16.826453 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:16.826484 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:16.826907 (+ 423us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:16.827099 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:16.827102 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:16.827587 (+ 485us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:16.827767 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:16.827771 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:16.828451 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:16.828452 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:16.829225 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:16.829227 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:16.829573 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:16.829574 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:16.829677 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:16.830048 (+ 371us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:16.830055 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:16.831297 (+ 1242us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:52:16.832541 (+ 1244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:16.832547 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:16.832550 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:16.832820 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:16.832996 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:16.832997 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:16.832997 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:16.833136 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:16.833369 (+ 233us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:52:16.833374 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:52:16.833522 (+ 148us) client_negotiation.cc:769] Sending connection context
0808 16:52:16.833606 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 16:52:16.833715 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":108,"threads_started":1}
I20260808 16:52:16.834244 13615 heartbeater.cc:347] Connected to a master server at 127.8.24.62:33683
I20260808 16:52:16.834316 13615 heartbeater.cc:464] Registering TS with master...
I20260808 16:52:16.834501 13615 heartbeater.cc:511] Master 127.8.24.62:33683 requested a full tablet report, sending...
I20260808 16:52:16.834825 12907 ts_manager.cc:194] Registered new tserver with Master: 4f67afeb23314b9bb17575080194a91d (127.8.24.3:45419)
I20260808 16:52:16.835170 12907 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.3@KRBTEST.COM'} at 127.8.24.3:57111
I20260808 16:52:16.844192 8288 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 16:52:16 dist-test-slave-6zbq krb5kdc[12836](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207936, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 08 16:52:16 dist-test-slave-6zbq krb5kdc[12836](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207936, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260808 16:52:16.865581 12975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:16.859575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42388 (local address 127.8.24.62:33683)
0808 16:52:16.859654 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:16.859656 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:16.859718 (+ 62us) server_negotiation.cc:407] Connection header received
0808 16:52:16.859765 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:16.859766 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:16.859774 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:16.859819 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:16.860325 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:16.860783 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:16.861443 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:16.861609 (+ 166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:16.861649 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:16.863607 (+ 1958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:16.863608 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:16.863609 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:16.863615 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:16.864790 (+ 1175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:16.865041 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:16.865042 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:16.865042 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:16.865086 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:16.865202 (+ 116us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:16.865202 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:16.865202 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:16.865327 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:16.865391 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:16.865459 (+ 68us) server_negotiation.cc:299] Negotiation successful
0808 16:52:16.865513 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
W20260808 16:52:16.865816 12907 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:42388
I20260808 16:52:16.866175 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 13000
I20260808 16:52:16.880515 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 13205
I20260808 16:52:16.887015 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 13411
I20260808 16:52:16.900399 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 12858
2026-08-08T16:52:16Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (1999 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786207919384338-8288-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 16:52:16 dist-test-slave-6zbq krb5kdc[13634](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 16:52:16 dist-test-slave-6zbq krb5kdc[13634](info): set up 2 sockets
Aug 08 16:52:16 dist-test-slave-6zbq krb5kdc[13634](info): commencing operation
krb5kdc: starting...
W20260808 16:52:18.032568 8288 mini_kdc.cc:121] Time spent starting KDC: real 1.108s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786207919384338-8288-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786207919384338-8288-0/krb5kdc/test-user.keytab.
Aug 08 16:52:18 dist-test-slave-6zbq krb5kdc[13634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207938, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T16:52:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T16:52:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.24.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
I20260808 16:52:18.137576 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786207919384338-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786207919384338-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786207919384338-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:45737
--webserver_interface=127.8.24.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:40729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:45737
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:52:18.214565 13654 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:52:18.214741 13654 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:52:18.214771 13654 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:52:18.215863 13654 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 16:52:18.215904 13654 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:52:18.215924 13654 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:52:18.215939 13654 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 16:52:18.215955 13654 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 16:52:18.217420 13654 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:40729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786207919384338-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786207919384338-8288-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:45737
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:45737
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.24.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13654
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786207919384338-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:52:18.217648 13654 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:52:18.217835 13654 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:52:18.222162 13668 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:18.222296 13654 server_base.cc:1034] running on GCE node
W20260808 16:52:18.222159 13665 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:18.222157 13664 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:18.222597 13654 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:52:18.222852 13654 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:52:18.223989 13654 hybrid_clock.cc:648] HybridClock initialized: now 1786207938223976 us; error 27 us; skew 500 ppm
Aug 08 16:52:18 dist-test-slave-6zbq krb5kdc[13634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207938, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:52:18.226171 13654 init.cc:377] Logged in from keytab as kudu/127.8.24.62@KRBTEST.COM (short username kudu)
I20260808 16:52:18.226583 13654 webserver.cc:460] Webserver started at http://127.8.24.62:44889/ using document root <none> and password file <none>
I20260808 16:52:18.226769 13654 fs_manager.cc:359] Metadata directory not provided
I20260808 16:52:18.226815 13654 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:52:18.226930 13654 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:52:18.227862 13654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786207919384338-8288-0/minicluster-data/master-0/data/instance:
uuid: "30eadf44526a4f81a0cb9272c46cf6bf"
format_stamp: "Formatted at 2026-08-08 16:52:18 on dist-test-slave-6zbq"
server_key: "a5e584ea35373e61cdd69bc8a444b477"
server_key_iv: "21bf3e8d9a2605b5916f1b0c1aaa7aeb"
server_key_version: "encryptionkey@0"
I20260808 16:52:18.228184 13654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786207919384338-8288-0/minicluster-data/master-0/wal/instance:
uuid: "30eadf44526a4f81a0cb9272c46cf6bf"
format_stamp: "Formatted at 2026-08-08 16:52:18 on dist-test-slave-6zbq"
server_key: "a5e584ea35373e61cdd69bc8a444b477"
server_key_iv: "21bf3e8d9a2605b5916f1b0c1aaa7aeb"
server_key_version: "encryptionkey@0"
I20260808 16:52:18.229338 13654 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 16:52:18.229939 13681 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:18.230086 13654 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 16:52:18.230175 13654 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786207919384338-8288-0/minicluster-data/master-0/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786207919384338-8288-0/minicluster-data/master-0/wal
uuid: "30eadf44526a4f81a0cb9272c46cf6bf"
format_stamp: "Formatted at 2026-08-08 16:52:18 on dist-test-slave-6zbq"
server_key: "a5e584ea35373e61cdd69bc8a444b477"
server_key_iv: "21bf3e8d9a2605b5916f1b0c1aaa7aeb"
server_key_version: "encryptionkey@0"
I20260808 16:52:18.230234 13654 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786207919384338-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786207919384338-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786207919384338-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:52:18.244357 13654 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:52:18.245838 13654 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:52:18.245934 13654 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:52:18.249562 13654 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.62:45737
I20260808 16:52:18.249591 13765 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.62:45737 every 8 connection(s)
I20260808 16:52:18.250079 13654 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
I20260808 16:52:18.250686 13766 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:52:18.251207 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 13654
I20260808 16:52:18.251291 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786207919384338-8288-0/minicluster-data/master-0/wal/instance
I20260808 16:52:18.251451 8288 external_mini_cluster.cc:1428] Setting key 8fcfaec01f1d144be7fcb1e28e6e9e5d
I20260808 16:52:18.253057 13766 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30eadf44526a4f81a0cb9272c46cf6bf: Bootstrap starting.
I20260808 16:52:18.253726 13766 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 30eadf44526a4f81a0cb9272c46cf6bf: Neither blocks nor log segments found. Creating new log.
I20260808 16:52:18.253996 13766 log.cc:824] T 00000000000000000000000000000000 P 30eadf44526a4f81a0cb9272c46cf6bf: Log is configured to *not* fsync() on all Append() calls
I20260808 16:52:18.254681 13766 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30eadf44526a4f81a0cb9272c46cf6bf: No bootstrap required, opened a new log
I20260808 16:52:18.256722 13766 raft_consensus.cc:348] T 00000000000000000000000000000000 P 30eadf44526a4f81a0cb9272c46cf6bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30eadf44526a4f81a0cb9272c46cf6bf" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 45737 } }
I20260808 16:52:18.256841 13766 raft_consensus.cc:374] T 00000000000000000000000000000000 P 30eadf44526a4f81a0cb9272c46cf6bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:52:18.256875 13766 raft_consensus.cc:729] T 00000000000000000000000000000000 P 30eadf44526a4f81a0cb9272c46cf6bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30eadf44526a4f81a0cb9272c46cf6bf, State: Initialized, Role: FOLLOWER
I20260808 16:52:18.256994 13766 consensus_queue.cc:260] T 00000000000000000000000000000000 P 30eadf44526a4f81a0cb9272c46cf6bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30eadf44526a4f81a0cb9272c46cf6bf" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 45737 } }
I20260808 16:52:18.257064 13766 raft_consensus.cc:388] T 00000000000000000000000000000000 P 30eadf44526a4f81a0cb9272c46cf6bf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 16:52:18.257095 13766 raft_consensus.cc:482] T 00000000000000000000000000000000 P 30eadf44526a4f81a0cb9272c46cf6bf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 16:52:18.257148 13766 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 30eadf44526a4f81a0cb9272c46cf6bf [term 0 FOLLOWER]: Advancing to term 1
Aug 08 16:52:18 dist-test-slave-6zbq krb5kdc[13634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207938, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260808 16:52:18.257593 13766 raft_consensus.cc:504] T 00000000000000000000000000000000 P 30eadf44526a4f81a0cb9272c46cf6bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30eadf44526a4f81a0cb9272c46cf6bf" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 45737 } }
I20260808 16:52:18.257687 13766 leader_election.cc:302] T 00000000000000000000000000000000 P 30eadf44526a4f81a0cb9272c46cf6bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 30eadf44526a4f81a0cb9272c46cf6bf; no voters:
I20260808 16:52:18.257819 13766 leader_election.cc:288] T 00000000000000000000000000000000 P 30eadf44526a4f81a0cb9272c46cf6bf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 16:52:18.257875 13772 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 30eadf44526a4f81a0cb9272c46cf6bf [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:52:18.258064 13772 raft_consensus.cc:686] T 00000000000000000000000000000000 P 30eadf44526a4f81a0cb9272c46cf6bf [term 1 LEADER]: Becoming Leader. State: Replica: 30eadf44526a4f81a0cb9272c46cf6bf, State: Running, Role: LEADER
I20260808 16:52:18.258052 13766 sys_catalog.cc:564] T 00000000000000000000000000000000 P 30eadf44526a4f81a0cb9272c46cf6bf [sys.catalog]: configured and running, proceeding with master startup.
I20260808 16:52:18.258213 13772 consensus_queue.cc:237] T 00000000000000000000000000000000 P 30eadf44526a4f81a0cb9272c46cf6bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30eadf44526a4f81a0cb9272c46cf6bf" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 45737 } }
I20260808 16:52:18.259236 13777 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30eadf44526a4f81a0cb9272c46cf6bf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "30eadf44526a4f81a0cb9272c46cf6bf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30eadf44526a4f81a0cb9272c46cf6bf" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 45737 } } }
I20260808 16:52:18.259323 13777 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30eadf44526a4f81a0cb9272c46cf6bf [sys.catalog]: This master's current role is: LEADER
I20260808 16:52:18.259691 13773 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30eadf44526a4f81a0cb9272c46cf6bf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 30eadf44526a4f81a0cb9272c46cf6bf. Latest consensus state: current_term: 1 leader_uuid: "30eadf44526a4f81a0cb9272c46cf6bf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30eadf44526a4f81a0cb9272c46cf6bf" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 45737 } } }
I20260808 16:52:18.259776 13773 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30eadf44526a4f81a0cb9272c46cf6bf [sys.catalog]: This master's current role is: LEADER
W20260808 16:52:18.259932 13799 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 30eadf44526a4f81a0cb9272c46cf6bf: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 16:52:18.259976 13799 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 16:52:18.260030 13797 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 16:52:18.260164 13797 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 16:52:18.261631 13769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:18.251877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43798 (local address 127.8.24.62:45737)
0808 16:52:18.252049 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:18.252056 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:18.252080 (+ 24us) server_negotiation.cc:407] Connection header received
0808 16:52:18.252467 (+ 387us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:18.252473 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:18.252523 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:18.252637 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:18.253239 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:18.254522 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:18.255253 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:18.255875 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:18.255985 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:18.258075 (+ 2090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:18.258079 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:18.258079 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:18.258090 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:18.260050 (+ 1960us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:18.260449 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:18.260451 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:18.260452 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:18.260529 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:18.260748 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:18.260749 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:18.260749 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:18.261020 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:18.261215 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:18.261326 (+ 111us) server_negotiation.cc:299] Negotiation successful
0808 16:52:18.261446 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":80,"threads_started":1}
I20260808 16:52:18.261999 13797 catalog_manager.cc:1347] Generated new cluster ID: 435dd2a99624467691cc4ce9df9179dd
I20260808 16:52:18.262051 13797 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 16:52:18.278644 13797 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 16:52:18.279099 13797 catalog_manager.cc:1504] Loading token signing keys...
I20260808 16:52:18.287614 13797 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 30eadf44526a4f81a0cb9272c46cf6bf: Generated new TSK 0
I20260808 16:52:18.287765 13797 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.24.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.1.keytab.
Entry for principal kudu/127.8.24.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.1.keytab.
I20260808 16:52:18.316150 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786207919384338-8288-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786207919384338-8288-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786207919384338-8288-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786207919384338-8288-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.1:0
--local_ip_for_outbound_sockets=127.8.24.1
--webserver_interface=127.8.24.1
--webserver_port=0
--tserver_master_addrs=127.8.24.62:45737
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.1.keytab
--principal=kudu/127.8.24.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:40729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:52:18.416112 13807 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:52:18.416288 13807 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:52:18.416309 13807 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:52:18.417404 13807 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:52:18.417444 13807 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:52:18.417469 13807 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.1
I20260808 16:52:18.418936 13807 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:40729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786207919384338-8288-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786207919384338-8288-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.1.keytab
--principal=kudu/127.8.24.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786207919384338-8288-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.24.1
--webserver_port=0
--tserver_master_addrs=127.8.24.62:45737
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13807
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.1
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786207919384338-8288-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:52:18.419147 13807 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:52:18.419340 13807 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:52:18.423830 13820 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:18.423806 13816 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:18.423857 13814 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:18.423928 13807 server_base.cc:1034] running on GCE node
I20260808 16:52:18.424171 13807 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:52:18.424386 13807 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:52:18.425516 13807 hybrid_clock.cc:648] HybridClock initialized: now 1786207938425503 us; error 30 us; skew 500 ppm
Aug 08 16:52:18 dist-test-slave-6zbq krb5kdc[13634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207938, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:52:18.427775 13807 init.cc:377] Logged in from keytab as kudu/127.8.24.1@KRBTEST.COM (short username kudu)
I20260808 16:52:18.428180 13807 webserver.cc:460] Webserver started at http://127.8.24.1:34025/ using document root <none> and password file <none>
I20260808 16:52:18.428361 13807 fs_manager.cc:359] Metadata directory not provided
I20260808 16:52:18.428401 13807 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:52:18.428517 13807 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:52:18.429445 13807 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786207919384338-8288-0/minicluster-data/ts-0/data/instance:
uuid: "0012c6df9a534854b23e013b1b949ba7"
format_stamp: "Formatted at 2026-08-08 16:52:18 on dist-test-slave-6zbq"
server_key: "17ad6c94e773fa148037f139819dcf51"
server_key_iv: "ad1ee35cc6ffb67793fd13fcafef6de4"
server_key_version: "encryptionkey@0"
I20260808 16:52:18.429761 13807 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786207919384338-8288-0/minicluster-data/ts-0/wal/instance:
uuid: "0012c6df9a534854b23e013b1b949ba7"
format_stamp: "Formatted at 2026-08-08 16:52:18 on dist-test-slave-6zbq"
server_key: "17ad6c94e773fa148037f139819dcf51"
server_key_iv: "ad1ee35cc6ffb67793fd13fcafef6de4"
server_key_version: "encryptionkey@0"
I20260808 16:52:18.431173 13807 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 16:52:18.431828 13837 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:18.431967 13807 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 16:52:18.432045 13807 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786207919384338-8288-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786207919384338-8288-0/minicluster-data/ts-0/wal
uuid: "0012c6df9a534854b23e013b1b949ba7"
format_stamp: "Formatted at 2026-08-08 16:52:18 on dist-test-slave-6zbq"
server_key: "17ad6c94e773fa148037f139819dcf51"
server_key_iv: "ad1ee35cc6ffb67793fd13fcafef6de4"
server_key_version: "encryptionkey@0"
I20260808 16:52:18.432097 13807 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786207919384338-8288-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786207919384338-8288-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786207919384338-8288-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:52:18.449342 13807 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:52:18.450901 13807 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:52:18.450995 13807 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:52:18.451151 13807 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:52:18.451416 13807 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:52:18.451445 13807 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:18.451464 13807 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:52:18.451479 13807 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:18.456497 13807 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.1:43107
I20260808 16:52:18.456553 14005 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.1:43107 every 8 connection(s)
I20260808 16:52:18.456797 13807 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786207919384338-8288-0/minicluster-data/ts-0/data/info.pb
I20260808 16:52:18.460845 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 13807
I20260808 16:52:18.460950 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786207919384338-8288-0/minicluster-data/ts-0/wal/instance
I20260808 16:52:18.461149 8288 external_mini_cluster.cc:1428] Setting key 3d8746becd59d03eaa1ddb13abb7e57b
Aug 08 16:52:18 dist-test-slave-6zbq krb5kdc[13634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207938, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.1@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260808 16:52:18.465406 13769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:18.457466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:39609 (local address 127.8.24.62:45737)
0808 16:52:18.457536 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:18.457538 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:18.457666 (+ 128us) server_negotiation.cc:407] Connection header received
0808 16:52:18.458571 (+ 905us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:18.458572 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:18.458586 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:18.458644 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:18.459214 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:18.459717 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:18.460656 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:18.460946 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:18.461012 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:18.462984 (+ 1972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:18.462985 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:18.462985 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:18.462992 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:18.464212 (+ 1220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:18.464567 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:18.464568 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:18.464569 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:18.464648 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:18.464853 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:18.464853 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:18.464854 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:18.464978 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:18.465040 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:18.465235 (+ 195us) server_negotiation.cc:299] Negotiation successful
0808 16:52:18.465318 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 16:52:18.465445 14010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:18.457397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:45737 (local address 127.8.24.1:39609)
0808 16:52:18.457534 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:18.457575 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:18.458480 (+ 905us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:18.458694 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:18.458696 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:18.458930 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:18.459125 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:18.459130 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:18.459795 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:18.459796 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:18.460568 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:18.460570 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:18.461050 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:18.461051 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:18.461168 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:18.461625 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:18.461632 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:18.462902 (+ 1270us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:52:18.464278 (+ 1376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:18.464284 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:18.464286 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:18.464485 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:18.464706 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:18.464706 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:18.464706 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:18.464799 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:18.465032 (+ 233us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:52:18.465034 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:52:18.465130 (+ 96us) client_negotiation.cc:769] Sending connection context
0808 16:52:18.465199 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 16:52:18.465307 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":59,"threads_started":1}
I20260808 16:52:18.465883 14006 heartbeater.cc:347] Connected to a master server at 127.8.24.62:45737
I20260808 16:52:18.465992 14006 heartbeater.cc:464] Registering TS with master...
I20260808 16:52:18.466179 14006 heartbeater.cc:511] Master 127.8.24.62:45737 requested a full tablet report, sending...
I20260808 16:52:18.466688 13708 ts_manager.cc:194] Registered new tserver with Master: 0012c6df9a534854b23e013b1b949ba7 (127.8.24.1:43107)
I20260808 16:52:18.467259 13708 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.1@KRBTEST.COM'} at 127.8.24.1:39609
WARNING: no policy specified for kudu/127.8.24.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.2.keytab.
Entry for principal kudu/127.8.24.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.2.keytab.
I20260808 16:52:18.484716 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786207919384338-8288-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786207919384338-8288-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786207919384338-8288-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786207919384338-8288-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.2:0
--local_ip_for_outbound_sockets=127.8.24.2
--webserver_interface=127.8.24.2
--webserver_port=0
--tserver_master_addrs=127.8.24.62:45737
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.2.keytab
--principal=kudu/127.8.24.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:40729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:52:18.562024 14015 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:52:18.562199 14015 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:52:18.562220 14015 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:52:18.563287 14015 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:52:18.563321 14015 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:52:18.563344 14015 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.2
I20260808 16:52:18.564730 14015 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:40729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786207919384338-8288-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786207919384338-8288-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.2.keytab
--principal=kudu/127.8.24.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786207919384338-8288-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.24.2
--webserver_port=0
--tserver_master_addrs=127.8.24.62:45737
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14015
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.2
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786207919384338-8288-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:52:18.564898 14015 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:52:18.565057 14015 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:52:18.569507 14027 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:18.569480 14033 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:18.569484 14029 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:18.569676 14015 server_base.cc:1034] running on GCE node
I20260808 16:52:18.569871 14015 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:52:18.570088 14015 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:52:18.571218 14015 hybrid_clock.cc:648] HybridClock initialized: now 1786207938571202 us; error 29 us; skew 500 ppm
Aug 08 16:52:18 dist-test-slave-6zbq krb5kdc[13634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207938, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:52:18.573388 14015 init.cc:377] Logged in from keytab as kudu/127.8.24.2@KRBTEST.COM (short username kudu)
I20260808 16:52:18.573760 14015 webserver.cc:460] Webserver started at http://127.8.24.2:34995/ using document root <none> and password file <none>
I20260808 16:52:18.573956 14015 fs_manager.cc:359] Metadata directory not provided
I20260808 16:52:18.574003 14015 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:52:18.574117 14015 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:52:18.575062 14015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786207919384338-8288-0/minicluster-data/ts-1/data/instance:
uuid: "032e1e466eca45c8be39d663db298ad8"
format_stamp: "Formatted at 2026-08-08 16:52:18 on dist-test-slave-6zbq"
server_key: "8dc70212ea58d3449f5aa1660d9c39a1"
server_key_iv: "dbc5bfb5327c7b139b7d30403086dac1"
server_key_version: "encryptionkey@0"
I20260808 16:52:18.575368 14015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786207919384338-8288-0/minicluster-data/ts-1/wal/instance:
uuid: "032e1e466eca45c8be39d663db298ad8"
format_stamp: "Formatted at 2026-08-08 16:52:18 on dist-test-slave-6zbq"
server_key: "8dc70212ea58d3449f5aa1660d9c39a1"
server_key_iv: "dbc5bfb5327c7b139b7d30403086dac1"
server_key_version: "encryptionkey@0"
I20260808 16:52:18.576500 14015 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 16:52:18.577121 14047 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:18.577265 14015 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 16:52:18.577342 14015 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786207919384338-8288-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786207919384338-8288-0/minicluster-data/ts-1/wal
uuid: "032e1e466eca45c8be39d663db298ad8"
format_stamp: "Formatted at 2026-08-08 16:52:18 on dist-test-slave-6zbq"
server_key: "8dc70212ea58d3449f5aa1660d9c39a1"
server_key_iv: "dbc5bfb5327c7b139b7d30403086dac1"
server_key_version: "encryptionkey@0"
I20260808 16:52:18.577406 14015 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786207919384338-8288-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786207919384338-8288-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786207919384338-8288-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:52:18.588747 14015 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:52:18.590276 14015 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:52:18.590451 14015 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:52:18.590639 14015 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:52:18.590893 14015 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:52:18.590924 14015 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:18.590950 14015 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:52:18.590966 14015 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:18.595983 14015 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.2:44041
I20260808 16:52:18.596041 14229 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.2:44041 every 8 connection(s)
I20260808 16:52:18.596272 14015 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786207919384338-8288-0/minicluster-data/ts-1/data/info.pb
I20260808 16:52:18.598519 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 14015
I20260808 16:52:18.598619 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786207919384338-8288-0/minicluster-data/ts-1/wal/instance
I20260808 16:52:18.598789 8288 external_mini_cluster.cc:1428] Setting key a7ed2838c072f96eb5708b4c27b6138b
Aug 08 16:52:18 dist-test-slave-6zbq krb5kdc[13634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207938, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.2@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260808 16:52:18.605199 13769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:18.596838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:51585 (local address 127.8.24.62:45737)
0808 16:52:18.596878 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:18.596880 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:18.597052 (+ 172us) server_negotiation.cc:407] Connection header received
0808 16:52:18.597839 (+ 787us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:18.597840 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:18.597848 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:18.597894 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:18.598456 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:18.599053 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:18.600060 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:18.600273 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:18.600329 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:18.602261 (+ 1932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:18.602262 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:18.602262 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:18.602268 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:18.603523 (+ 1255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:18.603980 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:18.603981 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:18.603981 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:18.604026 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:18.604321 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:18.604323 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:18.604323 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:18.604486 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:18.604560 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:18.604941 (+ 381us) server_negotiation.cc:299] Negotiation successful
0808 16:52:18.605060 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260808 16:52:18.605232 14232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:18.596815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:45737 (local address 127.8.24.2:51585)
0808 16:52:18.596957 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:18.596992 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:18.597752 (+ 760us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:18.597942 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:18.597945 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:18.598162 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:18.598367 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:18.598373 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:18.599142 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:18.599143 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:18.599972 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:18.599975 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:18.600343 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:18.600345 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:18.600474 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:18.600896 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:18.600904 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:18.602175 (+ 1271us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:52:18.603605 (+ 1430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:18.603613 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:18.603615 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:18.603911 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:18.604065 (+ 154us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:18.604066 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:18.604067 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:18.604212 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:18.604573 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:52:18.604580 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:52:18.604755 (+ 175us) client_negotiation.cc:769] Sending connection context
0808 16:52:18.604874 (+ 119us) client_negotiation.cc:240] Negotiation successful
0808 16:52:18.605028 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":65,"threads_started":1}
I20260808 16:52:18.606701 14230 heartbeater.cc:347] Connected to a master server at 127.8.24.62:45737
I20260808 16:52:18.606796 14230 heartbeater.cc:464] Registering TS with master...
I20260808 16:52:18.606963 14230 heartbeater.cc:511] Master 127.8.24.62:45737 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.8.24.3@KRBTEST.COM; defaulting to no policy
I20260808 16:52:18.607388 13708 ts_manager.cc:194] Registered new tserver with Master: 032e1e466eca45c8be39d663db298ad8 (127.8.24.2:44041)
I20260808 16:52:18.607786 13708 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.2@KRBTEST.COM'} at 127.8.24.2:51585
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.3.keytab.
Entry for principal kudu/127.8.24.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.3.keytab.
I20260808 16:52:18.622318 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786207919384338-8288-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786207919384338-8288-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786207919384338-8288-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786207919384338-8288-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.3:0
--local_ip_for_outbound_sockets=127.8.24.3
--webserver_interface=127.8.24.3
--webserver_port=0
--tserver_master_addrs=127.8.24.62:45737
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.3.keytab
--principal=kudu/127.8.24.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:40729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:52:18.699395 14240 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:52:18.699566 14240 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:52:18.699589 14240 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:52:18.700644 14240 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:52:18.700676 14240 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:52:18.700699 14240 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.3
I20260808 16:52:18.702109 14240 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:40729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786207919384338-8288-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786207919384338-8288-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.3.keytab
--principal=kudu/127.8.24.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786207919384338-8288-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.24.3
--webserver_port=0
--tserver_master_addrs=127.8.24.62:45737
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14240
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.3
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786207919384338-8288-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:52:18.702293 14240 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:52:18.702486 14240 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:52:18.706936 14250 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:18.706926 14248 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:18.706957 14240 server_base.cc:1034] running on GCE node
W20260808 16:52:18.706916 14246 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:18.707297 14240 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:52:18.707531 14240 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:52:18.708663 14240 hybrid_clock.cc:648] HybridClock initialized: now 1786207938708655 us; error 33 us; skew 500 ppm
Aug 08 16:52:18 dist-test-slave-6zbq krb5kdc[13634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207938, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:52:18.710943 14240 init.cc:377] Logged in from keytab as kudu/127.8.24.3@KRBTEST.COM (short username kudu)
I20260808 16:52:18.711304 14240 webserver.cc:460] Webserver started at http://127.8.24.3:35333/ using document root <none> and password file <none>
I20260808 16:52:18.711491 14240 fs_manager.cc:359] Metadata directory not provided
I20260808 16:52:18.711536 14240 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:52:18.711656 14240 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:52:18.712543 14240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786207919384338-8288-0/minicluster-data/ts-2/data/instance:
uuid: "f1404cc5f1194037abbb87d1e7acc2fc"
format_stamp: "Formatted at 2026-08-08 16:52:18 on dist-test-slave-6zbq"
server_key: "cfd1e71ee2b3608add6051d2bc0f2b70"
server_key_iv: "bfbb915b0712cd5108d6e88310b87272"
server_key_version: "encryptionkey@0"
I20260808 16:52:18.712853 14240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786207919384338-8288-0/minicluster-data/ts-2/wal/instance:
uuid: "f1404cc5f1194037abbb87d1e7acc2fc"
format_stamp: "Formatted at 2026-08-08 16:52:18 on dist-test-slave-6zbq"
server_key: "cfd1e71ee2b3608add6051d2bc0f2b70"
server_key_iv: "bfbb915b0712cd5108d6e88310b87272"
server_key_version: "encryptionkey@0"
I20260808 16:52:18.713953 14240 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 16:52:18.714576 14261 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:18.714710 14240 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 16:52:18.714787 14240 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786207919384338-8288-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786207919384338-8288-0/minicluster-data/ts-2/wal
uuid: "f1404cc5f1194037abbb87d1e7acc2fc"
format_stamp: "Formatted at 2026-08-08 16:52:18 on dist-test-slave-6zbq"
server_key: "cfd1e71ee2b3608add6051d2bc0f2b70"
server_key_iv: "bfbb915b0712cd5108d6e88310b87272"
server_key_version: "encryptionkey@0"
I20260808 16:52:18.714843 14240 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786207919384338-8288-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786207919384338-8288-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786207919384338-8288-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:52:18.723872 14240 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:52:18.725313 14240 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:52:18.725428 14240 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:52:18.725597 14240 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:52:18.725848 14240 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:52:18.725880 14240 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:18.725905 14240 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:52:18.725929 14240 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:18.731172 14240 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.3:35829
I20260808 16:52:18.731209 14441 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.3:35829 every 8 connection(s)
I20260808 16:52:18.731460 14240 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786207919384338-8288-0/minicluster-data/ts-2/data/info.pb
I20260808 16:52:18.736162 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 14240
I20260808 16:52:18.736281 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786207919384338-8288-0/minicluster-data/ts-2/wal/instance
I20260808 16:52:18.736470 8288 external_mini_cluster.cc:1428] Setting key e5fbcd34c8994aa0f74a7bf89625015a
Aug 08 16:52:18 dist-test-slave-6zbq krb5kdc[13634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207938, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.3@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260808 16:52:18.739941 13769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:18.732036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:45987 (local address 127.8.24.62:45737)
0808 16:52:18.732084 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:18.732085 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:18.732249 (+ 164us) server_negotiation.cc:407] Connection header received
0808 16:52:18.733052 (+ 803us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:18.733053 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:18.733066 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:18.733115 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:18.733674 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:18.734152 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:18.735158 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:18.735355 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:18.735409 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:18.737538 (+ 2129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:18.737539 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:18.737540 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:18.737547 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:18.738753 (+ 1206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:18.739107 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:18.739108 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:18.739108 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:18.739161 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:18.739383 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:18.739384 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:18.739384 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:18.739489 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:18.739542 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:18.739782 (+ 240us) server_negotiation.cc:299] Negotiation successful
0808 16:52:18.739856 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 16:52:18.740080 14444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:18.731996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:45737 (local address 127.8.24.3:45987)
0808 16:52:18.732133 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:18.732169 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:18.732956 (+ 787us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:18.733191 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:18.733195 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:18.733422 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:18.733587 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:18.733591 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:18.734228 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:18.734229 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:18.735077 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:18.735079 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:18.735422 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:18.735423 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:18.735535 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:18.735946 (+ 411us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:18.735954 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:18.737457 (+ 1503us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:52:18.738829 (+ 1372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:18.738835 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:18.738837 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:18.739040 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:18.739214 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:18.739215 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:18.739215 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:18.739326 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:18.739541 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:52:18.739547 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:52:18.739668 (+ 121us) client_negotiation.cc:769] Sending connection context
0808 16:52:18.739750 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 16:52:18.739896 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":76,"threads_started":1}
I20260808 16:52:18.740440 14442 heartbeater.cc:347] Connected to a master server at 127.8.24.62:45737
I20260808 16:52:18.740514 14442 heartbeater.cc:464] Registering TS with master...
I20260808 16:52:18.740636 14442 heartbeater.cc:511] Master 127.8.24.62:45737 requested a full tablet report, sending...
I20260808 16:52:18.740912 13708 ts_manager.cc:194] Registered new tserver with Master: f1404cc5f1194037abbb87d1e7acc2fc (127.8.24.3:35829)
I20260808 16:52:18.741240 13708 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.3@KRBTEST.COM'} at 127.8.24.3:45987
I20260808 16:52:18.747846 8288 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 16:52:18 dist-test-slave-6zbq krb5kdc[13634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207938, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 16:52:18 dist-test-slave-6zbq krb5kdc[13634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207938, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.24.1@KRBTEST.COM
I20260808 16:52:18.768838 14455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:18.762475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47460 (local address 127.8.24.1:43107)
0808 16:52:18.762613 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:18.762617 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:18.762625 (+ 8us) server_negotiation.cc:407] Connection header received
0808 16:52:18.762645 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:18.762647 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:18.762659 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:18.762721 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:18.763227 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:18.763896 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:18.764575 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:18.764869 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:18.764928 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:18.766796 (+ 1868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:18.766803 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:18.766803 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:18.766810 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:18.768008 (+ 1198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:18.768287 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:18.768289 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:18.768290 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:18.768339 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:18.768459 (+ 120us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:18.768459 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:18.768460 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:18.768564 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:18.768652 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:18.768704 (+ 52us) server_negotiation.cc:299] Negotiation successful
0808 16:52:18.768764 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":55,"threads_started":1}
Aug 08 16:52:18 dist-test-slave-6zbq krb5kdc[13634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207938, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260808 16:52:18.775336 13769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:18.769929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43806 (local address 127.8.24.62:45737)
0808 16:52:18.769966 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:18.769968 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:18.769976 (+ 8us) server_negotiation.cc:407] Connection header received
0808 16:52:18.770020 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:18.770020 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:18.770028 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:18.770067 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:18.770485 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:18.770916 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:18.771489 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:18.771653 (+ 164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:18.771692 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:18.773410 (+ 1718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:18.773411 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:18.773411 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:18.773417 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:18.774579 (+ 1162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:18.774823 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:18.774824 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:18.774824 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:18.774866 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:18.774977 (+ 111us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:18.774978 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:18.774978 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:18.775084 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:18.775145 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:18.775227 (+ 82us) server_negotiation.cc:299] Negotiation successful
0808 16:52:18.775282 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
W20260808 16:52:18.775498 13708 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43806
I20260808 16:52:18.775758 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 13807
I20260808 16:52:18.786767 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 14015
I20260808 16:52:18.797098 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 14240
I20260808 16:52:18.809406 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 13654
2026-08-08T16:52:18Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (1905 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786207919384338-8288-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 16:52:18 dist-test-slave-6zbq krb5kdc[14462](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 16:52:18 dist-test-slave-6zbq krb5kdc[14462](info): set up 2 sockets
Aug 08 16:52:18 dist-test-slave-6zbq krb5kdc[14462](info): commencing operation
krb5kdc: starting...
W20260808 16:52:19.936501 8288 mini_kdc.cc:121] Time spent starting KDC: real 1.107s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786207919384338-8288-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786207919384338-8288-0/krb5kdc/test-user.keytab.
Aug 08 16:52:20 dist-test-slave-6zbq krb5kdc[14462](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207940, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T16:52:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T16:52:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.24.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
I20260808 16:52:20.040637 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786207919384338-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786207919384338-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786207919384338-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:38501
--webserver_interface=127.8.24.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:32881
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:38501
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:52:20.117154 14484 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:52:20.117331 14484 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:52:20.117352 14484 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:52:20.118419 14484 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 16:52:20.118450 14484 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:52:20.118462 14484 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:52:20.118474 14484 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 16:52:20.118484 14484 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 16:52:20.119894 14484 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:32881
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786207919384338-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786207919384338-8288-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:38501
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:38501
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.24.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14484
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786207919384338-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:52:20.120069 14484 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:52:20.120222 14484 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:52:20.124465 14491 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:20.124433 14493 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:20.124480 14484 server_base.cc:1034] running on GCE node
W20260808 16:52:20.124450 14490 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:20.124887 14484 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:52:20.125139 14484 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:52:20.126281 14484 hybrid_clock.cc:648] HybridClock initialized: now 1786207940126267 us; error 28 us; skew 500 ppm
Aug 08 16:52:20 dist-test-slave-6zbq krb5kdc[14462](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207940, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:52:20.128437 14484 init.cc:377] Logged in from keytab as kudu/127.8.24.62@KRBTEST.COM (short username kudu)
I20260808 16:52:20.128752 14484 fs_manager.cc:359] Metadata directory not provided
I20260808 16:52:20.128803 14484 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:52:20.128919 14484 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:52:20.129865 14484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786207919384338-8288-0/minicluster-data/master-0/data/instance:
uuid: "37536edc6baa41ecb92bf74a023245de"
format_stamp: "Formatted at 2026-08-08 16:52:20 on dist-test-slave-6zbq"
server_key: "7efe43ec791c45697e049880fbdc31ef"
server_key_iv: "23f7706a8abd31dc0766210ea8971b95"
server_key_version: "encryptionkey@0"
I20260808 16:52:20.130173 14484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786207919384338-8288-0/minicluster-data/master-0/wal/instance:
uuid: "37536edc6baa41ecb92bf74a023245de"
format_stamp: "Formatted at 2026-08-08 16:52:20 on dist-test-slave-6zbq"
server_key: "7efe43ec791c45697e049880fbdc31ef"
server_key_iv: "23f7706a8abd31dc0766210ea8971b95"
server_key_version: "encryptionkey@0"
I20260808 16:52:20.131350 14484 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260808 16:52:20.131973 14500 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:20.132121 14484 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 16:52:20.132205 14484 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786207919384338-8288-0/minicluster-data/master-0/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786207919384338-8288-0/minicluster-data/master-0/wal
uuid: "37536edc6baa41ecb92bf74a023245de"
format_stamp: "Formatted at 2026-08-08 16:52:20 on dist-test-slave-6zbq"
server_key: "7efe43ec791c45697e049880fbdc31ef"
server_key_iv: "23f7706a8abd31dc0766210ea8971b95"
server_key_version: "encryptionkey@0"
I20260808 16:52:20.132264 14484 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786207919384338-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786207919384338-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786207919384338-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:52:20.148942 14484 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:52:20.150470 14484 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:52:20.150593 14484 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:52:20.154034 14484 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.62:38501
I20260808 16:52:20.154078 14597 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.62:38501 every 8 connection(s)
I20260808 16:52:20.154299 14484 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
I20260808 16:52:20.155323 14598 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:52:20.157341 14598 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 37536edc6baa41ecb92bf74a023245de: Bootstrap starting.
I20260808 16:52:20.157873 14598 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 37536edc6baa41ecb92bf74a023245de: Neither blocks nor log segments found. Creating new log.
I20260808 16:52:20.158138 14598 log.cc:824] T 00000000000000000000000000000000 P 37536edc6baa41ecb92bf74a023245de: Log is configured to *not* fsync() on all Append() calls
I20260808 16:52:20.158762 14598 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 37536edc6baa41ecb92bf74a023245de: No bootstrap required, opened a new log
I20260808 16:52:20.160229 14598 raft_consensus.cc:348] T 00000000000000000000000000000000 P 37536edc6baa41ecb92bf74a023245de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37536edc6baa41ecb92bf74a023245de" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 38501 } }
I20260808 16:52:20.160362 14598 raft_consensus.cc:374] T 00000000000000000000000000000000 P 37536edc6baa41ecb92bf74a023245de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:52:20.160396 14598 raft_consensus.cc:729] T 00000000000000000000000000000000 P 37536edc6baa41ecb92bf74a023245de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37536edc6baa41ecb92bf74a023245de, State: Initialized, Role: FOLLOWER
I20260808 16:52:20.160528 14598 consensus_queue.cc:260] T 00000000000000000000000000000000 P 37536edc6baa41ecb92bf74a023245de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37536edc6baa41ecb92bf74a023245de" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 38501 } }
I20260808 16:52:20.160588 14598 raft_consensus.cc:388] T 00000000000000000000000000000000 P 37536edc6baa41ecb92bf74a023245de [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 16:52:20.160609 14598 raft_consensus.cc:482] T 00000000000000000000000000000000 P 37536edc6baa41ecb92bf74a023245de [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 16:52:20.160652 14598 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 37536edc6baa41ecb92bf74a023245de [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:52:20.161383 14598 raft_consensus.cc:504] T 00000000000000000000000000000000 P 37536edc6baa41ecb92bf74a023245de [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37536edc6baa41ecb92bf74a023245de" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 38501 } }
I20260808 16:52:20.161474 14598 leader_election.cc:302] T 00000000000000000000000000000000 P 37536edc6baa41ecb92bf74a023245de [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 37536edc6baa41ecb92bf74a023245de; no voters:
I20260808 16:52:20.161620 14598 leader_election.cc:288] T 00000000000000000000000000000000 P 37536edc6baa41ecb92bf74a023245de [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 16:52:20.161666 14603 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 37536edc6baa41ecb92bf74a023245de [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:52:20.161837 14598 sys_catalog.cc:564] T 00000000000000000000000000000000 P 37536edc6baa41ecb92bf74a023245de [sys.catalog]: configured and running, proceeding with master startup.
I20260808 16:52:20.161825 14603 raft_consensus.cc:686] T 00000000000000000000000000000000 P 37536edc6baa41ecb92bf74a023245de [term 1 LEADER]: Becoming Leader. State: Replica: 37536edc6baa41ecb92bf74a023245de, State: Running, Role: LEADER
I20260808 16:52:20.161989 14603 consensus_queue.cc:237] T 00000000000000000000000000000000 P 37536edc6baa41ecb92bf74a023245de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37536edc6baa41ecb92bf74a023245de" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 38501 } }
I20260808 16:52:20.162364 14605 sys_catalog.cc:455] T 00000000000000000000000000000000 P 37536edc6baa41ecb92bf74a023245de [sys.catalog]: SysCatalogTable state changed. Reason: New leader 37536edc6baa41ecb92bf74a023245de. Latest consensus state: current_term: 1 leader_uuid: "37536edc6baa41ecb92bf74a023245de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37536edc6baa41ecb92bf74a023245de" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 38501 } } }
I20260808 16:52:20.162371 14604 sys_catalog.cc:455] T 00000000000000000000000000000000 P 37536edc6baa41ecb92bf74a023245de [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "37536edc6baa41ecb92bf74a023245de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37536edc6baa41ecb92bf74a023245de" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 38501 } } }
I20260808 16:52:20.162562 14604 sys_catalog.cc:458] T 00000000000000000000000000000000 P 37536edc6baa41ecb92bf74a023245de [sys.catalog]: This master's current role is: LEADER
I20260808 16:52:20.162549 14605 sys_catalog.cc:458] T 00000000000000000000000000000000 P 37536edc6baa41ecb92bf74a023245de [sys.catalog]: This master's current role is: LEADER
I20260808 16:52:20.162796 14617 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 16:52:20.163371 14617 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 16:52:20.164515 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 14484
I20260808 16:52:20.164615 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786207919384338-8288-0/minicluster-data/master-0/wal/instance
I20260808 16:52:20.164803 8288 external_mini_cluster.cc:1428] Setting key 54d469c653366f43542eb2aad1f61bc5
I20260808 16:52:20.164992 14617 catalog_manager.cc:1347] Generated new cluster ID: 02f0f586f8934cf190ffeb21637ab794
I20260808 16:52:20.165030 14617 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 16:52:20 dist-test-slave-6zbq krb5kdc[14462](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207940, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260808 16:52:20.175490 14633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:20.165376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37764 (local address 127.8.24.62:38501)
0808 16:52:20.165524 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:20.165532 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:20.165562 (+ 30us) server_negotiation.cc:407] Connection header received
0808 16:52:20.165671 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:20.165678 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:20.165732 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:20.165845 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:20.166560 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:20.167876 (+ 1316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:20.168908 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:20.169521 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:20.169610 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:20.172188 (+ 2578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:20.172192 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:20.172192 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:20.172203 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:20.173970 (+ 1767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:20.174327 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:20.174330 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:20.174330 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:20.174463 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:20.174632 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:20.174632 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:20.174633 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:20.174933 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:20.175081 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:20.175144 (+ 63us) server_negotiation.cc:299] Negotiation successful
0808 16:52:20.175271 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":104,"thread_start_us":71,"threads_started":1}
I20260808 16:52:20.176654 14617 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 16:52:20.177084 14617 catalog_manager.cc:1504] Loading token signing keys...
I20260808 16:52:20.180356 14617 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 37536edc6baa41ecb92bf74a023245de: Generated new TSK 0
I20260808 16:52:20.180482 14617 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.24.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.1.keytab.
Entry for principal kudu/127.8.24.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.1.keytab.
W20260808 16:52:20.292428 8288 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.8.24.1: real 0.111s user 0.000s sys 0.003s
I20260808 16:52:20.292815 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786207919384338-8288-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786207919384338-8288-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786207919384338-8288-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786207919384338-8288-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.1:0
--local_ip_for_outbound_sockets=127.8.24.1
--webserver_interface=127.8.24.1
--webserver_port=0
--tserver_master_addrs=127.8.24.62:38501
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.1.keytab
--principal=kudu/127.8.24.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:32881
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:52:20.371682 14639 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:52:20.371853 14639 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:52:20.371883 14639 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:52:20.372958 14639 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:52:20.372996 14639 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:52:20.373032 14639 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.1
I20260808 16:52:20.374464 14639 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:32881
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786207919384338-8288-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786207919384338-8288-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.1.keytab
--principal=kudu/127.8.24.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786207919384338-8288-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.24.1
--webserver_port=0
--tserver_master_addrs=127.8.24.62:38501
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14639
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.1
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786207919384338-8288-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:52:20.374665 14639 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:52:20.374852 14639 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:52:20.379328 14646 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:20.379308 14644 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:20.379346 14639 server_base.cc:1034] running on GCE node
W20260808 16:52:20.379321 14650 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:20.379701 14639 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:52:20.379925 14639 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:52:20.381047 14639 hybrid_clock.cc:648] HybridClock initialized: now 1786207940381039 us; error 26 us; skew 500 ppm
Aug 08 16:52:20 dist-test-slave-6zbq krb5kdc[14462](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207940, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:52:20.383361 14639 init.cc:377] Logged in from keytab as kudu/127.8.24.1@KRBTEST.COM (short username kudu)
I20260808 16:52:20.383618 14639 fs_manager.cc:359] Metadata directory not provided
I20260808 16:52:20.383659 14639 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:52:20.383741 14639 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:52:20.384624 14639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786207919384338-8288-0/minicluster-data/ts-0/data/instance:
uuid: "d58488e825384761b198f62cc97623a4"
format_stamp: "Formatted at 2026-08-08 16:52:20 on dist-test-slave-6zbq"
server_key: "e503cbe041a0914613f95d8959843537"
server_key_iv: "a8f7dd33d03f9fcfb6f53f968b454bd1"
server_key_version: "encryptionkey@0"
I20260808 16:52:20.384900 14639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786207919384338-8288-0/minicluster-data/ts-0/wal/instance:
uuid: "d58488e825384761b198f62cc97623a4"
format_stamp: "Formatted at 2026-08-08 16:52:20 on dist-test-slave-6zbq"
server_key: "e503cbe041a0914613f95d8959843537"
server_key_iv: "a8f7dd33d03f9fcfb6f53f968b454bd1"
server_key_version: "encryptionkey@0"
I20260808 16:52:20.385926 14639 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 16:52:20.386523 14661 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:20.386657 14639 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:20.386736 14639 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786207919384338-8288-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786207919384338-8288-0/minicluster-data/ts-0/wal
uuid: "d58488e825384761b198f62cc97623a4"
format_stamp: "Formatted at 2026-08-08 16:52:20 on dist-test-slave-6zbq"
server_key: "e503cbe041a0914613f95d8959843537"
server_key_iv: "a8f7dd33d03f9fcfb6f53f968b454bd1"
server_key_version: "encryptionkey@0"
I20260808 16:52:20.386806 14639 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786207919384338-8288-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786207919384338-8288-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786207919384338-8288-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:52:20.400149 14639 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:52:20.402055 14639 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:52:20.402173 14639 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:52:20.402374 14639 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:52:20.402652 14639 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:52:20.402683 14639 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:20.402709 14639 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:52:20.402734 14639 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:20.407868 14639 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.1:43827
I20260808 16:52:20.407924 14851 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.1:43827 every 8 connection(s)
I20260808 16:52:20.408121 14639 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786207919384338-8288-0/minicluster-data/ts-0/data/info.pb
Aug 08 16:52:20 dist-test-slave-6zbq krb5kdc[14462](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207940, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.1@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260808 16:52:20.415627 14633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:20.408682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:48297 (local address 127.8.24.62:38501)
0808 16:52:20.408731 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:20.408734 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:20.408935 (+ 201us) server_negotiation.cc:407] Connection header received
0808 16:52:20.409329 (+ 394us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:20.409330 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:20.409340 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:20.409387 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:20.410158 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:20.410692 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:20.411582 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:20.411788 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:20.411836 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:20.413566 (+ 1730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:20.413567 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:20.413567 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:20.413574 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:20.414658 (+ 1084us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:20.414986 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:20.414987 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:20.414987 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:20.415030 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:20.415195 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:20.415195 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:20.415195 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:20.415284 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:20.415329 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:20.415510 (+ 181us) server_negotiation.cc:299] Negotiation successful
0808 16:52:20.415567 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 16:52:20.415753 14854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:20.408678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:38501 (local address 127.8.24.1:48297)
0808 16:52:20.408819 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:20.408850 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:20.409249 (+ 399us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:20.409434 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:20.409437 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:20.409654 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:20.410092 (+ 438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:20.410096 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:20.410755 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:20.410756 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:20.411505 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:20.411507 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:20.411844 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:20.411844 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:20.411931 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:20.412319 (+ 388us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:20.412327 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:20.413490 (+ 1163us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:52:20.414727 (+ 1237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:20.414732 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:20.414734 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:20.414935 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:20.415071 (+ 136us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:20.415071 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:20.415071 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:20.415152 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:20.415326 (+ 174us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:52:20.415329 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:52:20.415424 (+ 95us) client_negotiation.cc:769] Sending connection context
0808 16:52:20.415494 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 16:52:20.415595 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":72,"threads_started":1}
I20260808 16:52:20.416203 14852 heartbeater.cc:347] Connected to a master server at 127.8.24.62:38501
I20260808 16:52:20.416277 14852 heartbeater.cc:464] Registering TS with master...
I20260808 16:52:20.416409 14852 heartbeater.cc:511] Master 127.8.24.62:38501 requested a full tablet report, sending...
I20260808 16:52:20.416824 14533 ts_manager.cc:194] Registered new tserver with Master: d58488e825384761b198f62cc97623a4 (127.8.24.1:43827)
I20260808 16:52:20.417377 14533 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.1@KRBTEST.COM'} at 127.8.24.1:48297
I20260808 16:52:20.417789 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 14639
I20260808 16:52:20.417876 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786207919384338-8288-0/minicluster-data/ts-0/wal/instance
I20260808 16:52:20.418037 8288 external_mini_cluster.cc:1428] Setting key cf29e1ca6b8abb6c39d377a373ae1f1d
WARNING: no policy specified for kudu/127.8.24.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.2.keytab.
Entry for principal kudu/127.8.24.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.2.keytab.
I20260808 16:52:20.438473 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786207919384338-8288-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786207919384338-8288-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786207919384338-8288-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786207919384338-8288-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.2:0
--local_ip_for_outbound_sockets=127.8.24.2
--webserver_interface=127.8.24.2
--webserver_port=0
--tserver_master_addrs=127.8.24.62:38501
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.2.keytab
--principal=kudu/127.8.24.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:32881
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:52:20.516515 14858 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:52:20.516685 14858 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:52:20.516716 14858 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:52:20.517823 14858 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:52:20.517861 14858 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:52:20.517896 14858 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.2
I20260808 16:52:20.519402 14858 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:32881
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786207919384338-8288-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786207919384338-8288-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.2.keytab
--principal=kudu/127.8.24.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786207919384338-8288-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.24.2
--webserver_port=0
--tserver_master_addrs=127.8.24.62:38501
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14858
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.2
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786207919384338-8288-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:52:20.519610 14858 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:52:20.519799 14858 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:52:20.524153 14870 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:20.524161 14866 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:20.524304 14858 server_base.cc:1034] running on GCE node
W20260808 16:52:20.524446 14867 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:20.524652 14858 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:52:20.524854 14858 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:52:20.525985 14858 hybrid_clock.cc:648] HybridClock initialized: now 1786207940525972 us; error 29 us; skew 500 ppm
Aug 08 16:52:20 dist-test-slave-6zbq krb5kdc[14462](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207940, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:52:20.528234 14858 init.cc:377] Logged in from keytab as kudu/127.8.24.2@KRBTEST.COM (short username kudu)
I20260808 16:52:20.528545 14858 fs_manager.cc:359] Metadata directory not provided
I20260808 16:52:20.528606 14858 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:52:20.528731 14858 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:52:20.529660 14858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786207919384338-8288-0/minicluster-data/ts-1/data/instance:
uuid: "64aa33e042fc45fbad91c08c25778066"
format_stamp: "Formatted at 2026-08-08 16:52:20 on dist-test-slave-6zbq"
server_key: "a6df5a1b5821e645a60fbf312c75041d"
server_key_iv: "b4517bc906b2f86332081b56036039ff"
server_key_version: "encryptionkey@0"
I20260808 16:52:20.529974 14858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786207919384338-8288-0/minicluster-data/ts-1/wal/instance:
uuid: "64aa33e042fc45fbad91c08c25778066"
format_stamp: "Formatted at 2026-08-08 16:52:20 on dist-test-slave-6zbq"
server_key: "a6df5a1b5821e645a60fbf312c75041d"
server_key_iv: "b4517bc906b2f86332081b56036039ff"
server_key_version: "encryptionkey@0"
I20260808 16:52:20.531219 14858 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 16:52:20.531821 14878 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:20.531973 14858 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 16:52:20.532032 14858 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786207919384338-8288-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786207919384338-8288-0/minicluster-data/ts-1/wal
uuid: "64aa33e042fc45fbad91c08c25778066"
format_stamp: "Formatted at 2026-08-08 16:52:20 on dist-test-slave-6zbq"
server_key: "a6df5a1b5821e645a60fbf312c75041d"
server_key_iv: "b4517bc906b2f86332081b56036039ff"
server_key_version: "encryptionkey@0"
I20260808 16:52:20.532096 14858 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786207919384338-8288-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786207919384338-8288-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786207919384338-8288-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:52:20.567178 14858 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:52:20.568849 14858 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:52:20.568958 14858 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:52:20.569190 14858 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:52:20.569463 14858 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:52:20.569495 14858 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:20.569515 14858 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:52:20.569538 14858 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:20.574867 14858 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.2:35057
I20260808 16:52:20.574900 15062 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.2:35057 every 8 connection(s)
I20260808 16:52:20.575114 14858 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786207919384338-8288-0/minicluster-data/ts-1/data/info.pb
Aug 08 16:52:20 dist-test-slave-6zbq krb5kdc[14462](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207940, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.2@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260808 16:52:20.582484 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 14858
I20260808 16:52:20.582594 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786207919384338-8288-0/minicluster-data/ts-1/wal/instance
I20260808 16:52:20.582814 8288 external_mini_cluster.cc:1428] Setting key 8cf57031720bcc6f8c25951b065f2e37
I20260808 16:52:20.582852 14633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:20.575703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:39187 (local address 127.8.24.62:38501)
0808 16:52:20.575746 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:20.575748 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:20.575918 (+ 170us) server_negotiation.cc:407] Connection header received
0808 16:52:20.576350 (+ 432us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:20.576352 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:20.576362 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:20.576413 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:20.577193 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:20.577683 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:20.578586 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:20.578772 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:20.578824 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:20.580518 (+ 1694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:20.580519 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:20.580520 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:20.580527 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:20.581738 (+ 1211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:20.582030 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:20.582030 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:20.582031 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:20.582075 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:20.582247 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:20.582247 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:20.582248 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:20.582388 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:20.582437 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:20.582721 (+ 284us) server_negotiation.cc:299] Negotiation successful
0808 16:52:20.582789 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 16:52:20.583045 15067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:20.575683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:38501 (local address 127.8.24.2:39187)
0808 16:52:20.575813 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:20.575844 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:20.576247 (+ 403us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:20.576460 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:20.576463 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:20.576682 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:20.577121 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:20.577125 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:20.577742 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:20.577743 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:20.578517 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:20.578519 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:20.578831 (+ 312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:20.578831 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:20.578914 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:20.579263 (+ 349us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:20.579271 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:20.580439 (+ 1168us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:52:20.581800 (+ 1361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:20.581802 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:20.581804 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:20.581978 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:20.582118 (+ 140us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:20.582118 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:20.582118 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:20.582207 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:20.582436 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:52:20.582442 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:52:20.582594 (+ 152us) client_negotiation.cc:769] Sending connection context
0808 16:52:20.582703 (+ 109us) client_negotiation.cc:240] Negotiation successful
0808 16:52:20.582851 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":72,"threads_started":1}
I20260808 16:52:20.583564 15064 heartbeater.cc:347] Connected to a master server at 127.8.24.62:38501
I20260808 16:52:20.583666 15064 heartbeater.cc:464] Registering TS with master...
I20260808 16:52:20.583801 15064 heartbeater.cc:511] Master 127.8.24.62:38501 requested a full tablet report, sending...
I20260808 16:52:20.584120 14533 ts_manager.cc:194] Registered new tserver with Master: 64aa33e042fc45fbad91c08c25778066 (127.8.24.2:35057)
I20260808 16:52:20.584471 14533 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.2@KRBTEST.COM'} at 127.8.24.2:39187
WARNING: no policy specified for kudu/127.8.24.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.3.keytab.
Entry for principal kudu/127.8.24.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.3.keytab.
I20260808 16:52:20.604979 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786207919384338-8288-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786207919384338-8288-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786207919384338-8288-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786207919384338-8288-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.3:0
--local_ip_for_outbound_sockets=127.8.24.3
--webserver_interface=127.8.24.3
--webserver_port=0
--tserver_master_addrs=127.8.24.62:38501
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.3.keytab
--principal=kudu/127.8.24.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:32881
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:52:20.690066 15073 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:52:20.690240 15073 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:52:20.690263 15073 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:52:20.691396 15073 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:52:20.691437 15073 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:52:20.691464 15073 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.3
I20260808 16:52:20.692903 15073 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:32881
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786207919384338-8288-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786207919384338-8288-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.3.keytab
--principal=kudu/127.8.24.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786207919384338-8288-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.24.3
--webserver_port=0
--tserver_master_addrs=127.8.24.62:38501
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15073
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.3
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786207919384338-8288-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:52:20.693097 15073 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:52:20.693284 15073 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:52:20.697949 15086 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:20.698084 15073 server_base.cc:1034] running on GCE node
W20260808 16:52:20.697938 15085 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:20.697938 15090 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:20.698406 15073 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:52:20.698629 15073 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:52:20.699759 15073 hybrid_clock.cc:648] HybridClock initialized: now 1786207940699743 us; error 29 us; skew 500 ppm
Aug 08 16:52:20 dist-test-slave-6zbq krb5kdc[14462](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207940, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:52:20.702082 15073 init.cc:377] Logged in from keytab as kudu/127.8.24.3@KRBTEST.COM (short username kudu)
I20260808 16:52:20.702385 15073 fs_manager.cc:359] Metadata directory not provided
I20260808 16:52:20.702438 15073 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:52:20.702558 15073 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:52:20.703500 15073 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786207919384338-8288-0/minicluster-data/ts-2/data/instance:
uuid: "8b217e2c3afd4557a8d04c2801a8330e"
format_stamp: "Formatted at 2026-08-08 16:52:20 on dist-test-slave-6zbq"
server_key: "9bd04b9c9ea5319799a99ea4df04cb8d"
server_key_iv: "2b9877ae8cc5a8401193cdd0ee90d173"
server_key_version: "encryptionkey@0"
I20260808 16:52:20.703814 15073 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786207919384338-8288-0/minicluster-data/ts-2/wal/instance:
uuid: "8b217e2c3afd4557a8d04c2801a8330e"
format_stamp: "Formatted at 2026-08-08 16:52:20 on dist-test-slave-6zbq"
server_key: "9bd04b9c9ea5319799a99ea4df04cb8d"
server_key_iv: "2b9877ae8cc5a8401193cdd0ee90d173"
server_key_version: "encryptionkey@0"
I20260808 16:52:20.704906 15073 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 16:52:20.705520 15098 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:20.705679 15073 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 16:52:20.705758 15073 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786207919384338-8288-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786207919384338-8288-0/minicluster-data/ts-2/wal
uuid: "8b217e2c3afd4557a8d04c2801a8330e"
format_stamp: "Formatted at 2026-08-08 16:52:20 on dist-test-slave-6zbq"
server_key: "9bd04b9c9ea5319799a99ea4df04cb8d"
server_key_iv: "2b9877ae8cc5a8401193cdd0ee90d173"
server_key_version: "encryptionkey@0"
I20260808 16:52:20.705811 15073 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786207919384338-8288-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786207919384338-8288-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786207919384338-8288-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:52:20.727684 15073 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:52:20.729240 15073 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:52:20.729336 15073 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:52:20.729496 15073 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:52:20.729725 15073 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:52:20.729755 15073 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:20.729773 15073 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:52:20.729789 15073 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:20.735296 15073 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.3:37899
I20260808 16:52:20.735344 15280 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.3:37899 every 8 connection(s)
I20260808 16:52:20.735592 15073 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786207919384338-8288-0/minicluster-data/ts-2/data/info.pb
I20260808 16:52:20.736073 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 15073
I20260808 16:52:20.736166 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786207919384338-8288-0/minicluster-data/ts-2/wal/instance
I20260808 16:52:20.736338 8288 external_mini_cluster.cc:1428] Setting key b1fa61b6b48f1bbdb383b48ef52ee1a7
Aug 08 16:52:20 dist-test-slave-6zbq krb5kdc[14462](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207940, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.3@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260808 16:52:20.743541 14633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:20.736210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:59745 (local address 127.8.24.62:38501)
0808 16:52:20.736256 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:20.736258 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:20.736667 (+ 409us) server_negotiation.cc:407] Connection header received
0808 16:52:20.737129 (+ 462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:20.737130 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:20.737139 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:20.737170 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:20.737966 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:20.738503 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:20.739405 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:20.739581 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:20.739623 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:20.741345 (+ 1722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:20.741346 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:20.741346 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:20.741353 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:20.742605 (+ 1252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:20.742909 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:20.742910 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:20.742910 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:20.742954 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:20.743114 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:20.743115 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:20.743115 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:20.743202 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:20.743244 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:20.743428 (+ 184us) server_negotiation.cc:299] Negotiation successful
0808 16:52:20.743480 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 16:52:20.743706 15290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:20.736147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:38501 (local address 127.8.24.3:59745)
0808 16:52:20.736551 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:20.736590 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:20.737045 (+ 455us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:20.737222 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:20.737225 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:20.737452 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:20.737901 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:20.737906 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:20.738571 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:20.738572 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:20.739344 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:20.739346 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:20.739632 (+ 286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:20.739632 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:20.739726 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:20.740072 (+ 346us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:20.740080 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:20.741266 (+ 1186us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:52:20.742665 (+ 1399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:20.742668 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:20.742670 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:20.742856 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:20.742994 (+ 138us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:20.742995 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:20.742995 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:20.743075 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:20.743241 (+ 166us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:52:20.743244 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:52:20.743341 (+ 97us) client_negotiation.cc:769] Sending connection context
0808 16:52:20.743413 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 16:52:20.743516 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":78,"threads_started":1}
I20260808 16:52:20.744104 15286 heartbeater.cc:347] Connected to a master server at 127.8.24.62:38501
I20260808 16:52:20.744186 15286 heartbeater.cc:464] Registering TS with master...
I20260808 16:52:20.744298 15286 heartbeater.cc:511] Master 127.8.24.62:38501 requested a full tablet report, sending...
I20260808 16:52:20.744634 14533 ts_manager.cc:194] Registered new tserver with Master: 8b217e2c3afd4557a8d04c2801a8330e (127.8.24.3:37899)
I20260808 16:52:20.744951 14533 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.3@KRBTEST.COM'} at 127.8.24.3:59745
I20260808 16:52:20.747712 8288 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 16:52:20.753708 14633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:20.748835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37770 (local address 127.8.24.62:38501)
0808 16:52:20.748899 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:20.748901 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:20.748951 (+ 50us) server_negotiation.cc:407] Connection header received
0808 16:52:20.749006 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:20.749007 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:20.749014 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:20.749049 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:20.749524 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:20.749949 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:20.750649 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:20.750821 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:20.750866 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:20.751818 (+ 952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:20.751819 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:20.751820 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:20.751825 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:20.752954 (+ 1129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:20.753214 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:20.753214 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:20.753214 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:20.753254 (+ 40us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:20.753372 (+ 118us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:20.753372 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:20.753372 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:20.753481 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:20.753544 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:20.753609 (+ 65us) server_negotiation.cc:299] Negotiation successful
0808 16:52:20.753657 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 16:52:20.754978 14533 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37770:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 16:52:20.756405 14533 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 16:52:20.762358 15306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:20.758430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.3:37899 (local address 127.0.0.1:45612)
0808 16:52:20.758993 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:20.759000 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:20.759038 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:20.759398 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:20.759399 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:20.759401 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:52:20.759471 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:20.759473 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:20.760611 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:20.760612 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:20.761446 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:20.761448 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:20.762063 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:20.762064 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:20.762140 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:20.762153 (+ 13us) client_negotiation.cc:769] Sending connection context
0808 16:52:20.762221 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 16:52:20.762270 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":551,"thread_start_us":76,"threads_started":1}
I20260808 16:52:20.762367 15309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:20.758604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45612 (local address 127.8.24.3:37899)
0808 16:52:20.758736 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:20.758741 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:20.759085 (+ 344us) server_negotiation.cc:407] Connection header received
0808 16:52:20.759125 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:20.759127 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:20.759258 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:20.759326 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:52:20.759546 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:20.760502 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:20.761507 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:20.761999 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:20.762062 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:20.762115 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:20.762237 (+ 122us) server_negotiation.cc:299] Negotiation successful
0808 16:52:20.762271 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":62,"threads_started":1}
I20260808 16:52:20.762717 15307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:20.758535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48740 (local address 127.8.24.2:35057)
0808 16:52:20.758685 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:20.758690 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:20.758957 (+ 267us) server_negotiation.cc:407] Connection header received
0808 16:52:20.759001 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:20.759003 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:20.759018 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:20.759092 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:52:20.759203 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:20.760273 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:20.761515 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:20.762258 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:20.762298 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:20.762365 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:20.762596 (+ 231us) server_negotiation.cc:299] Negotiation successful
0808 16:52:20.762639 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":86,"threads_started":1}
I20260808 16:52:20.762923 15305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:20.758425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.2:35057 (local address 127.0.0.1:48740)
0808 16:52:20.758894 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:20.758901 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:20.758935 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:20.759094 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:20.759095 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:20.759097 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:52:20.759149 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:20.759151 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:20.760373 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:20.760374 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:20.761423 (+ 1049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:20.761425 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:20.762449 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:20.762450 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:20.762516 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:20.762518 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 16:52:20.762815 (+ 297us) client_negotiation.cc:240] Negotiation successful
0808 16:52:20.762854 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":95,"threads_started":1}
I20260808 16:52:20.762923 15304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:20.758410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:43827 (local address 127.0.0.1:35754)
0808 16:52:20.758716 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:20.758755 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:20.758814 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:20.758990 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:20.758992 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:20.758994 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:52:20.759055 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:20.759056 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:20.760055 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:20.760056 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:20.761265 (+ 1209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:20.761269 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:20.762279 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:20.762280 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:20.762385 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:20.762389 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 16:52:20.762693 (+ 304us) client_negotiation.cc:240] Negotiation successful
0808 16:52:20.762736 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":49,"threads_started":1}
I20260808 16:52:20.763022 15171 tablet_service.cc:1467] Processing CreateTablet for tablet 70eb0f9d32df4ba8a80d5ac745afd823 (DEFAULT_TABLE table=test-table [id=b828ab9560ee4913957eaabe5e21d456]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:52:20.763273 15310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:20.758645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35754 (local address 127.8.24.1:43827)
0808 16:52:20.758785 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:20.758789 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:20.758799 (+ 10us) server_negotiation.cc:407] Connection header received
0808 16:52:20.758882 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:20.758883 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:20.758895 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:20.758972 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:52:20.759150 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:20.759974 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:20.761361 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:20.762120 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:20.762186 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:20.762242 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:20.763152 (+ 910us) server_negotiation.cc:299] Negotiation successful
0808 16:52:20.763195 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":86,"threads_started":1}
I20260808 16:52:20.763396 15171 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 70eb0f9d32df4ba8a80d5ac745afd823. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:52:20.763628 14958 tablet_service.cc:1467] Processing CreateTablet for tablet 70eb0f9d32df4ba8a80d5ac745afd823 (DEFAULT_TABLE table=test-table [id=b828ab9560ee4913957eaabe5e21d456]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:52:20.763942 14958 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 70eb0f9d32df4ba8a80d5ac745afd823. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:52:20.766757 15312 tablet_bootstrap.cc:492] T 70eb0f9d32df4ba8a80d5ac745afd823 P 8b217e2c3afd4557a8d04c2801a8330e: Bootstrap starting.
I20260808 16:52:20.767060 15313 tablet_bootstrap.cc:492] T 70eb0f9d32df4ba8a80d5ac745afd823 P 64aa33e042fc45fbad91c08c25778066: Bootstrap starting.
I20260808 16:52:20.766964 14737 tablet_service.cc:1467] Processing CreateTablet for tablet 70eb0f9d32df4ba8a80d5ac745afd823 (DEFAULT_TABLE table=test-table [id=b828ab9560ee4913957eaabe5e21d456]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:52:20.767295 14737 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 70eb0f9d32df4ba8a80d5ac745afd823. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:52:20.767395 15312 tablet_bootstrap.cc:654] T 70eb0f9d32df4ba8a80d5ac745afd823 P 8b217e2c3afd4557a8d04c2801a8330e: Neither blocks nor log segments found. Creating new log.
I20260808 16:52:20.767760 15313 tablet_bootstrap.cc:654] T 70eb0f9d32df4ba8a80d5ac745afd823 P 64aa33e042fc45fbad91c08c25778066: Neither blocks nor log segments found. Creating new log.
I20260808 16:52:20.767976 15312 log.cc:824] T 70eb0f9d32df4ba8a80d5ac745afd823 P 8b217e2c3afd4557a8d04c2801a8330e: Log is configured to *not* fsync() on all Append() calls
I20260808 16:52:20.768308 15313 log.cc:824] T 70eb0f9d32df4ba8a80d5ac745afd823 P 64aa33e042fc45fbad91c08c25778066: Log is configured to *not* fsync() on all Append() calls
I20260808 16:52:20.768661 15312 tablet_bootstrap.cc:492] T 70eb0f9d32df4ba8a80d5ac745afd823 P 8b217e2c3afd4557a8d04c2801a8330e: No bootstrap required, opened a new log
I20260808 16:52:20.768736 15312 ts_tablet_manager.cc:1397] T 70eb0f9d32df4ba8a80d5ac745afd823 P 8b217e2c3afd4557a8d04c2801a8330e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 16:52:20.769083 15313 tablet_bootstrap.cc:492] T 70eb0f9d32df4ba8a80d5ac745afd823 P 64aa33e042fc45fbad91c08c25778066: No bootstrap required, opened a new log
I20260808 16:52:20.769161 15313 ts_tablet_manager.cc:1397] T 70eb0f9d32df4ba8a80d5ac745afd823 P 64aa33e042fc45fbad91c08c25778066: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 16:52:20.770238 15312 raft_consensus.cc:348] T 70eb0f9d32df4ba8a80d5ac745afd823 P 8b217e2c3afd4557a8d04c2801a8330e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d58488e825384761b198f62cc97623a4" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 43827 } } peers { permanent_uuid: "64aa33e042fc45fbad91c08c25778066" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 35057 } } peers { permanent_uuid: "8b217e2c3afd4557a8d04c2801a8330e" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 37899 } }
I20260808 16:52:20.770517 15312 raft_consensus.cc:374] T 70eb0f9d32df4ba8a80d5ac745afd823 P 8b217e2c3afd4557a8d04c2801a8330e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:52:20.770578 15312 raft_consensus.cc:729] T 70eb0f9d32df4ba8a80d5ac745afd823 P 8b217e2c3afd4557a8d04c2801a8330e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b217e2c3afd4557a8d04c2801a8330e, State: Initialized, Role: FOLLOWER
I20260808 16:52:20.770545 15318 tablet_bootstrap.cc:492] T 70eb0f9d32df4ba8a80d5ac745afd823 P d58488e825384761b198f62cc97623a4: Bootstrap starting.
I20260808 16:52:20.770715 15312 consensus_queue.cc:260] T 70eb0f9d32df4ba8a80d5ac745afd823 P 8b217e2c3afd4557a8d04c2801a8330e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d58488e825384761b198f62cc97623a4" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 43827 } } peers { permanent_uuid: "64aa33e042fc45fbad91c08c25778066" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 35057 } } peers { permanent_uuid: "8b217e2c3afd4557a8d04c2801a8330e" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 37899 } }
I20260808 16:52:20.771018 15286 heartbeater.cc:503] Master 127.8.24.62:38501 was elected leader, sending a full tablet report...
I20260808 16:52:20.771047 15313 raft_consensus.cc:348] T 70eb0f9d32df4ba8a80d5ac745afd823 P 64aa33e042fc45fbad91c08c25778066 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d58488e825384761b198f62cc97623a4" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 43827 } } peers { permanent_uuid: "64aa33e042fc45fbad91c08c25778066" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 35057 } } peers { permanent_uuid: "8b217e2c3afd4557a8d04c2801a8330e" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 37899 } }
I20260808 16:52:20.771198 15313 raft_consensus.cc:374] T 70eb0f9d32df4ba8a80d5ac745afd823 P 64aa33e042fc45fbad91c08c25778066 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:52:20.771238 15313 raft_consensus.cc:729] T 70eb0f9d32df4ba8a80d5ac745afd823 P 64aa33e042fc45fbad91c08c25778066 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64aa33e042fc45fbad91c08c25778066, State: Initialized, Role: FOLLOWER
I20260808 16:52:20.771013 15312 ts_tablet_manager.cc:1428] T 70eb0f9d32df4ba8a80d5ac745afd823 P 8b217e2c3afd4557a8d04c2801a8330e: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 16:52:20.771380 15318 tablet_bootstrap.cc:654] T 70eb0f9d32df4ba8a80d5ac745afd823 P d58488e825384761b198f62cc97623a4: Neither blocks nor log segments found. Creating new log.
I20260808 16:52:20.771363 15313 consensus_queue.cc:260] T 70eb0f9d32df4ba8a80d5ac745afd823 P 64aa33e042fc45fbad91c08c25778066 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d58488e825384761b198f62cc97623a4" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 43827 } } peers { permanent_uuid: "64aa33e042fc45fbad91c08c25778066" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 35057 } } peers { permanent_uuid: "8b217e2c3afd4557a8d04c2801a8330e" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 37899 } }
I20260808 16:52:20.771647 15064 heartbeater.cc:503] Master 127.8.24.62:38501 was elected leader, sending a full tablet report...
I20260808 16:52:20.771859 15313 ts_tablet_manager.cc:1428] T 70eb0f9d32df4ba8a80d5ac745afd823 P 64aa33e042fc45fbad91c08c25778066: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 16:52:20.771919 15318 log.cc:824] T 70eb0f9d32df4ba8a80d5ac745afd823 P d58488e825384761b198f62cc97623a4: Log is configured to *not* fsync() on all Append() calls
I20260808 16:52:20.772600 15318 tablet_bootstrap.cc:492] T 70eb0f9d32df4ba8a80d5ac745afd823 P d58488e825384761b198f62cc97623a4: No bootstrap required, opened a new log
I20260808 16:52:20.772665 15318 ts_tablet_manager.cc:1397] T 70eb0f9d32df4ba8a80d5ac745afd823 P d58488e825384761b198f62cc97623a4: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 16:52:20.773986 15318 raft_consensus.cc:348] T 70eb0f9d32df4ba8a80d5ac745afd823 P d58488e825384761b198f62cc97623a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d58488e825384761b198f62cc97623a4" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 43827 } } peers { permanent_uuid: "64aa33e042fc45fbad91c08c25778066" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 35057 } } peers { permanent_uuid: "8b217e2c3afd4557a8d04c2801a8330e" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 37899 } }
I20260808 16:52:20.774112 15318 raft_consensus.cc:374] T 70eb0f9d32df4ba8a80d5ac745afd823 P d58488e825384761b198f62cc97623a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:52:20.774143 15318 raft_consensus.cc:729] T 70eb0f9d32df4ba8a80d5ac745afd823 P d58488e825384761b198f62cc97623a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d58488e825384761b198f62cc97623a4, State: Initialized, Role: FOLLOWER
I20260808 16:52:20.774242 15318 consensus_queue.cc:260] T 70eb0f9d32df4ba8a80d5ac745afd823 P d58488e825384761b198f62cc97623a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d58488e825384761b198f62cc97623a4" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 43827 } } peers { permanent_uuid: "64aa33e042fc45fbad91c08c25778066" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 35057 } } peers { permanent_uuid: "8b217e2c3afd4557a8d04c2801a8330e" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 37899 } }
I20260808 16:52:20.774483 15318 ts_tablet_manager.cc:1428] T 70eb0f9d32df4ba8a80d5ac745afd823 P d58488e825384761b198f62cc97623a4: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 16:52:20.774505 14852 heartbeater.cc:503] Master 127.8.24.62:38501 was elected leader, sending a full tablet report...
W20260808 16:52:20.825587 15065 tablet.cc:2369] T 70eb0f9d32df4ba8a80d5ac745afd823 P 64aa33e042fc45fbad91c08c25778066: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 16:52:20.908696 14853 tablet.cc:2369] T 70eb0f9d32df4ba8a80d5ac745afd823 P d58488e825384761b198f62cc97623a4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 16:52:20.986080 15287 tablet.cc:2369] T 70eb0f9d32df4ba8a80d5ac745afd823 P 8b217e2c3afd4557a8d04c2801a8330e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 16:52:21.045078 15324 raft_consensus.cc:482] T 70eb0f9d32df4ba8a80d5ac745afd823 P d58488e825384761b198f62cc97623a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 16:52:21.045177 15324 raft_consensus.cc:504] T 70eb0f9d32df4ba8a80d5ac745afd823 P d58488e825384761b198f62cc97623a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d58488e825384761b198f62cc97623a4" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 43827 } } peers { permanent_uuid: "64aa33e042fc45fbad91c08c25778066" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 35057 } } peers { permanent_uuid: "8b217e2c3afd4557a8d04c2801a8330e" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 37899 } }
I20260808 16:52:21.045523 15324 leader_election.cc:288] T 70eb0f9d32df4ba8a80d5ac745afd823 P d58488e825384761b198f62cc97623a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64aa33e042fc45fbad91c08c25778066 (127.8.24.2:35057), 8b217e2c3afd4557a8d04c2801a8330e (127.8.24.3:37899)
I20260808 16:52:21.048292 15307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:21.045743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:41453 (local address 127.8.24.2:35057)
0808 16:52:21.045800 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:21.045802 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:21.045872 (+ 70us) server_negotiation.cc:407] Connection header received
0808 16:52:21.045927 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:21.045928 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:21.045939 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:21.045989 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:52:21.046152 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:21.046613 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:21.047564 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:21.047951 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:21.048005 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:21.048014 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:21.048193 (+ 179us) server_negotiation.cc:299] Negotiation successful
0808 16:52:21.048229 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 16:52:21.048316 15309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:21.045793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:56033 (local address 127.8.24.3:37899)
0808 16:52:21.045834 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:21.045835 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:21.045861 (+ 26us) server_negotiation.cc:407] Connection header received
0808 16:52:21.045940 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:21.045940 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:21.045950 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:21.045997 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:52:21.046139 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:21.046639 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:21.047628 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:21.047990 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:21.048057 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:21.048067 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:21.048213 (+ 146us) server_negotiation.cc:299] Negotiation successful
0808 16:52:21.048248 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 16:52:21.048346 15329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:21.045669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.3:37899 (local address 127.8.24.1:56033)
0808 16:52:21.045805 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:21.045814 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:21.045877 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:21.046000 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:21.046000 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:21.046002 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:52:21.046083 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:21.046084 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:21.046723 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:21.046724 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:21.047549 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:21.047552 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:21.048074 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:21.048075 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:21.048151 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:21.048155 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 16:52:21.048206 (+ 51us) client_negotiation.cc:240] Negotiation successful
0808 16:52:21.048243 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"mutex_wait_us":5,"thread_start_us":62,"threads_started":1}
I20260808 16:52:21.048357 15328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:21.045658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.2:35057 (local address 127.8.24.1:41453)
0808 16:52:21.045799 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:21.045810 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:21.045874 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:21.045999 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:21.046000 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:21.046002 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:52:21.046090 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:21.046091 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:21.046676 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:21.046677 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:21.047488 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:21.047490 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:21.048023 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:21.048025 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:21.048112 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:21.048118 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 16:52:21.048183 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 16:52:21.048221 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"mutex_wait_us":22,"thread_start_us":78,"threads_started":1}
I20260808 16:52:21.048667 15208 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70eb0f9d32df4ba8a80d5ac745afd823" candidate_uuid: "d58488e825384761b198f62cc97623a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b217e2c3afd4557a8d04c2801a8330e" is_pre_election: true
I20260808 16:52:21.048802 15208 raft_consensus.cc:2454] T 70eb0f9d32df4ba8a80d5ac745afd823 P 8b217e2c3afd4557a8d04c2801a8330e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d58488e825384761b198f62cc97623a4 in term 0.
I20260808 16:52:21.048756 14993 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70eb0f9d32df4ba8a80d5ac745afd823" candidate_uuid: "d58488e825384761b198f62cc97623a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64aa33e042fc45fbad91c08c25778066" is_pre_election: true
I20260808 16:52:21.048904 14993 raft_consensus.cc:2454] T 70eb0f9d32df4ba8a80d5ac745afd823 P 64aa33e042fc45fbad91c08c25778066 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d58488e825384761b198f62cc97623a4 in term 0.
I20260808 16:52:21.049023 14665 leader_election.cc:302] T 70eb0f9d32df4ba8a80d5ac745afd823 P d58488e825384761b198f62cc97623a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b217e2c3afd4557a8d04c2801a8330e, d58488e825384761b198f62cc97623a4; no voters:
I20260808 16:52:21.049176 15324 raft_consensus.cc:2781] T 70eb0f9d32df4ba8a80d5ac745afd823 P d58488e825384761b198f62cc97623a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 16:52:21.049244 15324 raft_consensus.cc:482] T 70eb0f9d32df4ba8a80d5ac745afd823 P d58488e825384761b198f62cc97623a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 16:52:21.049260 15324 raft_consensus.cc:3037] T 70eb0f9d32df4ba8a80d5ac745afd823 P d58488e825384761b198f62cc97623a4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:52:21.049911 15324 raft_consensus.cc:504] T 70eb0f9d32df4ba8a80d5ac745afd823 P d58488e825384761b198f62cc97623a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d58488e825384761b198f62cc97623a4" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 43827 } } peers { permanent_uuid: "64aa33e042fc45fbad91c08c25778066" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 35057 } } peers { permanent_uuid: "8b217e2c3afd4557a8d04c2801a8330e" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 37899 } }
I20260808 16:52:21.050025 15324 leader_election.cc:288] T 70eb0f9d32df4ba8a80d5ac745afd823 P d58488e825384761b198f62cc97623a4 [CANDIDATE]: Term 1 election: Requested vote from peers 64aa33e042fc45fbad91c08c25778066 (127.8.24.2:35057), 8b217e2c3afd4557a8d04c2801a8330e (127.8.24.3:37899)
I20260808 16:52:21.050138 14993 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70eb0f9d32df4ba8a80d5ac745afd823" candidate_uuid: "d58488e825384761b198f62cc97623a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64aa33e042fc45fbad91c08c25778066"
I20260808 16:52:21.050170 15208 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70eb0f9d32df4ba8a80d5ac745afd823" candidate_uuid: "d58488e825384761b198f62cc97623a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b217e2c3afd4557a8d04c2801a8330e"
I20260808 16:52:21.050226 15208 raft_consensus.cc:3037] T 70eb0f9d32df4ba8a80d5ac745afd823 P 8b217e2c3afd4557a8d04c2801a8330e [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:52:21.050226 14993 raft_consensus.cc:3037] T 70eb0f9d32df4ba8a80d5ac745afd823 P 64aa33e042fc45fbad91c08c25778066 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:52:21.050915 15208 raft_consensus.cc:2454] T 70eb0f9d32df4ba8a80d5ac745afd823 P 8b217e2c3afd4557a8d04c2801a8330e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d58488e825384761b198f62cc97623a4 in term 1.
I20260808 16:52:21.050943 14993 raft_consensus.cc:2454] T 70eb0f9d32df4ba8a80d5ac745afd823 P 64aa33e042fc45fbad91c08c25778066 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d58488e825384761b198f62cc97623a4 in term 1.
I20260808 16:52:21.051079 14665 leader_election.cc:302] T 70eb0f9d32df4ba8a80d5ac745afd823 P d58488e825384761b198f62cc97623a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 64aa33e042fc45fbad91c08c25778066, 8b217e2c3afd4557a8d04c2801a8330e, d58488e825384761b198f62cc97623a4; no voters:
I20260808 16:52:21.051177 15324 raft_consensus.cc:2781] T 70eb0f9d32df4ba8a80d5ac745afd823 P d58488e825384761b198f62cc97623a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:52:21.051316 15324 raft_consensus.cc:686] T 70eb0f9d32df4ba8a80d5ac745afd823 P d58488e825384761b198f62cc97623a4 [term 1 LEADER]: Becoming Leader. State: Replica: d58488e825384761b198f62cc97623a4, State: Running, Role: LEADER
I20260808 16:52:21.051405 15324 consensus_queue.cc:237] T 70eb0f9d32df4ba8a80d5ac745afd823 P d58488e825384761b198f62cc97623a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d58488e825384761b198f62cc97623a4" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 43827 } } peers { permanent_uuid: "64aa33e042fc45fbad91c08c25778066" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 35057 } } peers { permanent_uuid: "8b217e2c3afd4557a8d04c2801a8330e" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 37899 } }
I20260808 16:52:21.052126 14532 catalog_manager.cc:5495] T 70eb0f9d32df4ba8a80d5ac745afd823 P d58488e825384761b198f62cc97623a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to d58488e825384761b198f62cc97623a4 (127.8.24.1). New cstate: current_term: 1 leader_uuid: "d58488e825384761b198f62cc97623a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d58488e825384761b198f62cc97623a4" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 43827 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64aa33e042fc45fbad91c08c25778066" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 35057 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b217e2c3afd4557a8d04c2801a8330e" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 37899 } health_report { overall_health: UNKNOWN } } }
I20260808 16:52:21.110900 15310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:21.108926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35762 (local address 127.8.24.1:43827)
0808 16:52:21.108966 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:21.108968 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:21.108975 (+ 7us) server_negotiation.cc:407] Connection header received
0808 16:52:21.109029 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:21.109030 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:21.109046 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:21.109099 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 16:52:21.109196 (+ 97us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:21.109612 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:21.110283 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:21.110466 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:21.110509 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:21.110671 (+ 162us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 16:52:21.110761 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 16:52:21.110813 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:21.110828 (+ 15us) server_negotiation.cc:299] Negotiation successful
0808 16:52:21.110842 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 16:52:21.112475 15208 raft_consensus.cc:1261] T 70eb0f9d32df4ba8a80d5ac745afd823 P 8b217e2c3afd4557a8d04c2801a8330e [term 1 FOLLOWER]: Refusing update from remote peer d58488e825384761b198f62cc97623a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:52:21.112476 14993 raft_consensus.cc:1261] T 70eb0f9d32df4ba8a80d5ac745afd823 P 64aa33e042fc45fbad91c08c25778066 [term 1 FOLLOWER]: Refusing update from remote peer d58488e825384761b198f62cc97623a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:52:21.112772 15330 consensus_queue.cc:1035] T 70eb0f9d32df4ba8a80d5ac745afd823 P d58488e825384761b198f62cc97623a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b217e2c3afd4557a8d04c2801a8330e" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 37899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:52:21.112902 15324 consensus_queue.cc:1035] T 70eb0f9d32df4ba8a80d5ac745afd823 P d58488e825384761b198f62cc97623a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64aa33e042fc45fbad91c08c25778066" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 35057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:52:21.114867 15341 mvcc.cc:205] Tried to move back new op lower bound from 7316307726794289152 to 7316307726547087360. Current Snapshot: MvccSnapshot[applied={T|T < 7316307726794289152}]
I20260808 16:52:21.117869 14533 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37770:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 16:52:21.117952 14533 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37770:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 16:52:21.118643 14533 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 37536edc6baa41ecb92bf74a023245de: Sending DeleteTablet for 3 replicas of tablet 70eb0f9d32df4ba8a80d5ac745afd823
I20260808 16:52:21.118955 14958 tablet_service.cc:1514] Processing DeleteTablet for tablet 70eb0f9d32df4ba8a80d5ac745afd823 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:52:21 UTC) from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:48740
I20260808 16:52:21.119068 15171 tablet_service.cc:1514] Processing DeleteTablet for tablet 70eb0f9d32df4ba8a80d5ac745afd823 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:52:21 UTC) from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:45612
I20260808 16:52:21.119174 14737 tablet_service.cc:1514] Processing DeleteTablet for tablet 70eb0f9d32df4ba8a80d5ac745afd823 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:52:21 UTC) from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:35754
I20260808 16:52:21.119282 15354 tablet_replica.cc:330] T 70eb0f9d32df4ba8a80d5ac745afd823 P 64aa33e042fc45fbad91c08c25778066: stopping tablet replica
I20260808 16:52:21.119378 15354 raft_consensus.cc:2229] T 70eb0f9d32df4ba8a80d5ac745afd823 P 64aa33e042fc45fbad91c08c25778066 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 16:52:21.119397 15356 tablet_replica.cc:330] T 70eb0f9d32df4ba8a80d5ac745afd823 P d58488e825384761b198f62cc97623a4: stopping tablet replica
I20260808 16:52:21.119479 15354 raft_consensus.cc:2258] T 70eb0f9d32df4ba8a80d5ac745afd823 P 64aa33e042fc45fbad91c08c25778066 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:52:21.119525 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 14639
I20260808 16:52:21.119625 15356 raft_consensus.cc:2229] T 70eb0f9d32df4ba8a80d5ac745afd823 P d58488e825384761b198f62cc97623a4 [term 1 LEADER]: Raft consensus shutting down.
I20260808 16:52:21.119774 15356 raft_consensus.cc:2258] T 70eb0f9d32df4ba8a80d5ac745afd823 P d58488e825384761b198f62cc97623a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:52:21.119892 15354 ts_tablet_manager.cc:1905] T 70eb0f9d32df4ba8a80d5ac745afd823 P 64aa33e042fc45fbad91c08c25778066: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:52:21.121688 15354 ts_tablet_manager.cc:1918] T 70eb0f9d32df4ba8a80d5ac745afd823 P 64aa33e042fc45fbad91c08c25778066: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 16:52:21.121776 15354 log.cc:1196] T 70eb0f9d32df4ba8a80d5ac745afd823 P 64aa33e042fc45fbad91c08c25778066: Deleting WAL directory at /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786207919384338-8288-0/minicluster-data/ts-1/wal/wals/70eb0f9d32df4ba8a80d5ac745afd823
I20260808 16:52:21.122082 15354 ts_tablet_manager.cc:1939] T 70eb0f9d32df4ba8a80d5ac745afd823 P 64aa33e042fc45fbad91c08c25778066: Deleting consensus metadata
I20260808 16:52:21.122540 14512 catalog_manager.cc:4841] TS 64aa33e042fc45fbad91c08c25778066 (127.8.24.2:35057): tablet 70eb0f9d32df4ba8a80d5ac745afd823 (table test-table [id=b828ab9560ee4913957eaabe5e21d456]) successfully deleted
I20260808 16:52:21.126281 15355 tablet_replica.cc:330] T 70eb0f9d32df4ba8a80d5ac745afd823 P 8b217e2c3afd4557a8d04c2801a8330e: stopping tablet replica
I20260808 16:52:21.126397 15355 raft_consensus.cc:2229] T 70eb0f9d32df4ba8a80d5ac745afd823 P 8b217e2c3afd4557a8d04c2801a8330e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 16:52:21.126490 15355 raft_consensus.cc:2258] T 70eb0f9d32df4ba8a80d5ac745afd823 P 8b217e2c3afd4557a8d04c2801a8330e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:52:21.126905 15355 ts_tablet_manager.cc:1905] T 70eb0f9d32df4ba8a80d5ac745afd823 P 8b217e2c3afd4557a8d04c2801a8330e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:52:21.128283 15355 ts_tablet_manager.cc:1918] T 70eb0f9d32df4ba8a80d5ac745afd823 P 8b217e2c3afd4557a8d04c2801a8330e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 16:52:21.128350 15355 log.cc:1196] T 70eb0f9d32df4ba8a80d5ac745afd823 P 8b217e2c3afd4557a8d04c2801a8330e: Deleting WAL directory at /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786207919384338-8288-0/minicluster-data/ts-2/wal/wals/70eb0f9d32df4ba8a80d5ac745afd823
I20260808 16:52:21.128577 15355 ts_tablet_manager.cc:1939] T 70eb0f9d32df4ba8a80d5ac745afd823 P 8b217e2c3afd4557a8d04c2801a8330e: Deleting consensus metadata
I20260808 16:52:21.128940 14503 catalog_manager.cc:4841] TS 8b217e2c3afd4557a8d04c2801a8330e (127.8.24.3:37899): tablet 70eb0f9d32df4ba8a80d5ac745afd823 (table test-table [id=b828ab9560ee4913957eaabe5e21d456]) successfully deleted
W20260808 16:52:21.131129 14502 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.24.1:43827 (error 108)
I20260808 16:52:21.131469 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 14858
I20260808 16:52:21.131574 15304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:21.131458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:43827 (local address 127.0.0.1:35772)
0808 16:52:21.131510 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:21.131545 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.1:43827: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260808 16:52:21.131675 14502 catalog_manager.cc:4584] TS d58488e825384761b198f62cc97623a4 (127.8.24.1:43827): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 70eb0f9d32df4ba8a80d5ac745afd823: Network error: Client connection negotiation failed: client connection to 127.8.24.1:43827: connect: Connection refused (error 111)
I20260808 16:52:21.139884 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 15073
I20260808 16:52:21.145555 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 14484
2026-08-08T16:52:21Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2339 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-08T16:52:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T16:52:21Z Disabled control of system clock
I20260808 16:52:21.176538 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786207919384338-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786207919384338-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786207919384338-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:37541
--webserver_interface=127.8.24.62
--webserver_port=0
--builtin_ntp_servers=127.8.24.20:41463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:37541
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 16:52:21.253563 15363 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:52:21.253743 15363 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:52:21.253763 15363 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:52:21.254889 15363 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 16:52:21.254926 15363 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:52:21.254938 15363 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:52:21.254961 15363 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 16:52:21.254983 15363 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 16:52:21.256287 15363 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:41463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786207919384338-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786207919384338-8288-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:37541
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:37541
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.24.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15363
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786207919384338-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:52:21.256495 15363 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:52:21.256757 15363 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:52:21.260957 15371 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:21.260974 15374 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:21.260962 15377 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:21.261178 15363 server_base.cc:1034] running on GCE node
I20260808 16:52:21.261379 15363 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:52:21.261616 15363 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:52:21.262753 15363 hybrid_clock.cc:648] HybridClock initialized: now 1786207941262738 us; error 29 us; skew 500 ppm
I20260808 16:52:21.263794 15363 webserver.cc:460] Webserver started at http://127.8.24.62:43207/ using document root <none> and password file <none>
I20260808 16:52:21.263993 15363 fs_manager.cc:359] Metadata directory not provided
I20260808 16:52:21.264034 15363 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:52:21.264146 15363 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:52:21.265064 15363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786207919384338-8288-0/minicluster-data/master-0/data/instance:
uuid: "1523edb278ab40ec961207d0e615810f"
format_stamp: "Formatted at 2026-08-08 16:52:21 on dist-test-slave-6zbq"
server_key: "6efa3ac9036f9f8b0b92a5542c4a1e3f"
server_key_iv: "dd22a81568a5e0fe09be66068b9efa66"
server_key_version: "encryptionkey@0"
I20260808 16:52:21.265370 15363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786207919384338-8288-0/minicluster-data/master-0/wal/instance:
uuid: "1523edb278ab40ec961207d0e615810f"
format_stamp: "Formatted at 2026-08-08 16:52:21 on dist-test-slave-6zbq"
server_key: "6efa3ac9036f9f8b0b92a5542c4a1e3f"
server_key_iv: "dd22a81568a5e0fe09be66068b9efa66"
server_key_version: "encryptionkey@0"
I20260808 16:52:21.266535 15363 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 16:52:21.267163 15389 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:21.267287 15363 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 16:52:21.267359 15363 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786207919384338-8288-0/minicluster-data/master-0/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786207919384338-8288-0/minicluster-data/master-0/wal
uuid: "1523edb278ab40ec961207d0e615810f"
format_stamp: "Formatted at 2026-08-08 16:52:21 on dist-test-slave-6zbq"
server_key: "6efa3ac9036f9f8b0b92a5542c4a1e3f"
server_key_iv: "dd22a81568a5e0fe09be66068b9efa66"
server_key_version: "encryptionkey@0"
I20260808 16:52:21.267422 15363 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786207919384338-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786207919384338-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786207919384338-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:52:21.270504 15363 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:52:21.270759 15363 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:52:21.270862 15363 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:52:21.274384 15363 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.62:37541
I20260808 16:52:21.274421 15463 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.62:37541 every 8 connection(s)
I20260808 16:52:21.274703 15363 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
I20260808 16:52:21.275261 15464 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:52:21.277801 15464 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1523edb278ab40ec961207d0e615810f: Bootstrap starting.
I20260808 16:52:21.278329 15464 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1523edb278ab40ec961207d0e615810f: Neither blocks nor log segments found. Creating new log.
I20260808 16:52:21.278626 15464 log.cc:824] T 00000000000000000000000000000000 P 1523edb278ab40ec961207d0e615810f: Log is configured to *not* fsync() on all Append() calls
I20260808 16:52:21.279193 15464 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1523edb278ab40ec961207d0e615810f: No bootstrap required, opened a new log
I20260808 16:52:21.280135 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 15363
I20260808 16:52:21.280237 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786207919384338-8288-0/minicluster-data/master-0/wal/instance
I20260808 16:52:21.280403 8288 external_mini_cluster.cc:1428] Setting key 44d010e32945b5a121b88f7e06603415
I20260808 16:52:21.280968 15464 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1523edb278ab40ec961207d0e615810f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1523edb278ab40ec961207d0e615810f" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 37541 } }
I20260808 16:52:21.281108 15464 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1523edb278ab40ec961207d0e615810f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:52:21.281131 15464 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1523edb278ab40ec961207d0e615810f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1523edb278ab40ec961207d0e615810f, State: Initialized, Role: FOLLOWER
I20260808 16:52:21.281246 15464 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1523edb278ab40ec961207d0e615810f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1523edb278ab40ec961207d0e615810f" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 37541 } }
I20260808 16:52:21.281312 15464 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1523edb278ab40ec961207d0e615810f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 16:52:21.281339 15464 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1523edb278ab40ec961207d0e615810f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 16:52:21.281383 15464 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1523edb278ab40ec961207d0e615810f [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:52:21.282002 15464 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1523edb278ab40ec961207d0e615810f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1523edb278ab40ec961207d0e615810f" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 37541 } }
I20260808 16:52:21.282107 15464 leader_election.cc:302] T 00000000000000000000000000000000 P 1523edb278ab40ec961207d0e615810f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1523edb278ab40ec961207d0e615810f; no voters:
I20260808 16:52:21.282258 15464 leader_election.cc:288] T 00000000000000000000000000000000 P 1523edb278ab40ec961207d0e615810f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 16:52:21.282305 15471 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1523edb278ab40ec961207d0e615810f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:52:21.282480 15471 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1523edb278ab40ec961207d0e615810f [term 1 LEADER]: Becoming Leader. State: Replica: 1523edb278ab40ec961207d0e615810f, State: Running, Role: LEADER
I20260808 16:52:21.282562 15464 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1523edb278ab40ec961207d0e615810f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 16:52:21.282581 15471 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1523edb278ab40ec961207d0e615810f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1523edb278ab40ec961207d0e615810f" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 37541 } }
I20260808 16:52:21.282729 15470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:21.280864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53598 (local address 127.8.24.62:37541)
0808 16:52:21.281047 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:21.281053 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:21.281075 (+ 22us) server_negotiation.cc:407] Connection header received
0808 16:52:21.281190 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:21.281197 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:21.281238 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:21.281371 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:21.281957 (+ 586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:21.281960 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:21.281960 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 16:52:21.281963 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:21.281995 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 16:52:21.282005 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:21.282055 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:21.282178 (+ 123us) server_negotiation.cc:299] Negotiation successful
0808 16:52:21.282238 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":70,"threads_started":1}
I20260808 16:52:21.282877 15471 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1523edb278ab40ec961207d0e615810f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1523edb278ab40ec961207d0e615810f. Latest consensus state: current_term: 1 leader_uuid: "1523edb278ab40ec961207d0e615810f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1523edb278ab40ec961207d0e615810f" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 37541 } } }
I20260808 16:52:21.282954 15471 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1523edb278ab40ec961207d0e615810f [sys.catalog]: This master's current role is: LEADER
I20260808 16:52:21.282920 15475 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1523edb278ab40ec961207d0e615810f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1523edb278ab40ec961207d0e615810f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1523edb278ab40ec961207d0e615810f" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 37541 } } }
I20260808 16:52:21.283124 15475 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1523edb278ab40ec961207d0e615810f [sys.catalog]: This master's current role is: LEADER
W20260808 16:52:21.284219 15496 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1523edb278ab40ec961207d0e615810f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 16:52:21.284263 15496 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 16:52:21.284335 15482 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 16:52:21.284534 15482 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 16:52:21.285975 15482 catalog_manager.cc:1347] Generated new cluster ID: a94ac7e564d942baab7cd6ff561ab4df
I20260808 16:52:21.286023 15482 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 16:52:21.328087 15482 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 16:52:21.328246 15482 catalog_manager.cc:1504] Loading token signing keys...
I20260808 16:52:21.333523 15482 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1523edb278ab40ec961207d0e615810f: Generated new TSK 0
I20260808 16:52:21.333665 15482 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 16:52:21.344219 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786207919384338-8288-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786207919384338-8288-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786207919384338-8288-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786207919384338-8288-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.1:0
--local_ip_for_outbound_sockets=127.8.24.1
--webserver_interface=127.8.24.1
--webserver_port=0
--tserver_master_addrs=127.8.24.62:37541
--builtin_ntp_servers=127.8.24.20:41463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 16:52:21.423388 15501 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:52:21.423563 15501 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:52:21.423584 15501 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:52:21.424702 15501 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:52:21.424738 15501 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:52:21.424762 15501 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.1
I20260808 16:52:21.426028 15501 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:41463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786207919384338-8288-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786207919384338-8288-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786207919384338-8288-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.24.1
--webserver_port=0
--tserver_master_addrs=127.8.24.62:37541
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15501
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.1
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786207919384338-8288-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:52:21.426224 15501 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:52:21.426493 15501 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:52:21.431001 15511 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:21.430975 15512 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:21.430984 15514 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:21.431074 15501 server_base.cc:1034] running on GCE node
I20260808 16:52:21.431348 15501 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:52:21.431548 15501 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:52:21.432686 15501 hybrid_clock.cc:648] HybridClock initialized: now 1786207941432673 us; error 29 us; skew 500 ppm
I20260808 16:52:21.433825 15501 webserver.cc:460] Webserver started at http://127.8.24.1:34521/ using document root <none> and password file <none>
I20260808 16:52:21.434017 15501 fs_manager.cc:359] Metadata directory not provided
I20260808 16:52:21.434064 15501 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:52:21.434197 15501 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:52:21.435185 15501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786207919384338-8288-0/minicluster-data/ts-0/data/instance:
uuid: "3de19bfd237d4931a04aad8b15204456"
format_stamp: "Formatted at 2026-08-08 16:52:21 on dist-test-slave-6zbq"
server_key: "6687f15a582831e95c832803b243e452"
server_key_iv: "4abfbaddbb87a67cc466de8dab4a777f"
server_key_version: "encryptionkey@0"
I20260808 16:52:21.435542 15501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786207919384338-8288-0/minicluster-data/ts-0/wal/instance:
uuid: "3de19bfd237d4931a04aad8b15204456"
format_stamp: "Formatted at 2026-08-08 16:52:21 on dist-test-slave-6zbq"
server_key: "6687f15a582831e95c832803b243e452"
server_key_iv: "4abfbaddbb87a67cc466de8dab4a777f"
server_key_version: "encryptionkey@0"
I20260808 16:52:21.436691 15501 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 16:52:21.437297 15525 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:21.437439 15501 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 16:52:21.437517 15501 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786207919384338-8288-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786207919384338-8288-0/minicluster-data/ts-0/wal
uuid: "3de19bfd237d4931a04aad8b15204456"
format_stamp: "Formatted at 2026-08-08 16:52:21 on dist-test-slave-6zbq"
server_key: "6687f15a582831e95c832803b243e452"
server_key_iv: "4abfbaddbb87a67cc466de8dab4a777f"
server_key_version: "encryptionkey@0"
I20260808 16:52:21.437582 15501 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786207919384338-8288-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786207919384338-8288-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786207919384338-8288-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:52:21.440673 15501 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:52:21.440889 15501 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:52:21.441004 15501 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:52:21.441211 15501 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:52:21.441479 15501 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:52:21.441510 15501 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:21.441535 15501 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:52:21.441552 15501 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:21.446805 15501 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.1:45505
I20260808 16:52:21.446869 15691 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.1:45505 every 8 connection(s)
I20260808 16:52:21.447108 15501 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786207919384338-8288-0/minicluster-data/ts-0/data/info.pb
I20260808 16:52:21.448982 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 15501
I20260808 16:52:21.449057 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786207919384338-8288-0/minicluster-data/ts-0/wal/instance
I20260808 16:52:21.449230 8288 external_mini_cluster.cc:1428] Setting key 4caddb7072021bc376a902299869ce78
I20260808 16:52:21.449576 15470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:21.447692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:37929 (local address 127.8.24.62:37541)
0808 16:52:21.447733 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:21.447735 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:21.447947 (+ 212us) server_negotiation.cc:407] Connection header received
0808 16:52:21.448321 (+ 374us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:21.448322 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:21.448328 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:21.448375 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:21.449338 (+ 963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:21.449339 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:21.449340 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 16:52:21.449344 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:21.449363 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 16:52:21.449368 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:21.449419 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:21.449491 (+ 72us) server_negotiation.cc:299] Negotiation successful
0808 16:52:21.449507 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 16:52:21.449761 15695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:21.447663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:37541 (local address 127.8.24.1:37929)
0808 16:52:21.447838 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:21.447874 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:21.448252 (+ 378us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:21.448420 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:21.448423 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:21.448430 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:21.449208 (+ 778us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 16:52:21.449208 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:21.449233 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 16:52:21.449259 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:52:21.449421 (+ 162us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:52:21.449426 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:52:21.449430 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 16:52:21.449491 (+ 61us) client_negotiation.cc:240] Negotiation successful
0808 16:52:21.449548 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":71,"threads_started":1}
I20260808 16:52:21.450065 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786207919384338-8288-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786207919384338-8288-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786207919384338-8288-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786207919384338-8288-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.2:0
--local_ip_for_outbound_sockets=127.8.24.2
--webserver_interface=127.8.24.2
--webserver_port=0
--tserver_master_addrs=127.8.24.62:37541
--builtin_ntp_servers=127.8.24.20:41463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260808 16:52:21.450146 15692 heartbeater.cc:347] Connected to a master server at 127.8.24.62:37541
I20260808 16:52:21.450224 15692 heartbeater.cc:464] Registering TS with master...
I20260808 16:52:21.450363 15692 heartbeater.cc:511] Master 127.8.24.62:37541 requested a full tablet report, sending...
I20260808 16:52:21.450816 15414 ts_manager.cc:194] Registered new tserver with Master: 3de19bfd237d4931a04aad8b15204456 (127.8.24.1:45505)
W20260808 16:52:21.528738 15698 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:52:21.528908 15698 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:52:21.528928 15698 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:52:21.530015 15698 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:52:21.530051 15698 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:52:21.530077 15698 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.2
I20260808 16:52:21.531376 15698 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:41463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786207919384338-8288-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786207919384338-8288-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786207919384338-8288-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.24.2
--webserver_port=0
--tserver_master_addrs=127.8.24.62:37541
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15698
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.2
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786207919384338-8288-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:52:21.531570 15698 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:52:21.531821 15698 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:52:21.536228 15704 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:21.536235 15706 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:21.536252 15709 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:21.536373 15698 server_base.cc:1034] running on GCE node
I20260808 16:52:21.536554 15698 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:52:21.536777 15698 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:52:21.537912 15698 hybrid_clock.cc:648] HybridClock initialized: now 1786207941537898 us; error 30 us; skew 500 ppm
I20260808 16:52:21.538986 15698 webserver.cc:460] Webserver started at http://127.8.24.2:41275/ using document root <none> and password file <none>
I20260808 16:52:21.539179 15698 fs_manager.cc:359] Metadata directory not provided
I20260808 16:52:21.539227 15698 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:52:21.539346 15698 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:52:21.540334 15698 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786207919384338-8288-0/minicluster-data/ts-1/data/instance:
uuid: "2d2bd967f0f64fc3a9a5853787f772e0"
format_stamp: "Formatted at 2026-08-08 16:52:21 on dist-test-slave-6zbq"
server_key: "20d300c3e2acf2035a530b71562ff259"
server_key_iv: "d3fb94eb50a58871b39c21056ef0fba6"
server_key_version: "encryptionkey@0"
I20260808 16:52:21.540647 15698 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786207919384338-8288-0/minicluster-data/ts-1/wal/instance:
uuid: "2d2bd967f0f64fc3a9a5853787f772e0"
format_stamp: "Formatted at 2026-08-08 16:52:21 on dist-test-slave-6zbq"
server_key: "20d300c3e2acf2035a530b71562ff259"
server_key_iv: "d3fb94eb50a58871b39c21056ef0fba6"
server_key_version: "encryptionkey@0"
I20260808 16:52:21.541814 15698 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 16:52:21.542433 15721 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:21.542567 15698 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 16:52:21.542655 15698 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786207919384338-8288-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786207919384338-8288-0/minicluster-data/ts-1/wal
uuid: "2d2bd967f0f64fc3a9a5853787f772e0"
format_stamp: "Formatted at 2026-08-08 16:52:21 on dist-test-slave-6zbq"
server_key: "20d300c3e2acf2035a530b71562ff259"
server_key_iv: "d3fb94eb50a58871b39c21056ef0fba6"
server_key_version: "encryptionkey@0"
I20260808 16:52:21.542719 15698 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786207919384338-8288-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786207919384338-8288-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786207919384338-8288-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:52:21.545697 15698 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:52:21.545917 15698 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:52:21.546028 15698 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:52:21.546204 15698 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:52:21.546510 15698 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:52:21.546545 15698 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:21.546569 15698 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:52:21.546593 15698 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:21.551715 15698 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.2:37745
I20260808 16:52:21.551769 15918 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.2:37745 every 8 connection(s)
I20260808 16:52:21.552014 15698 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786207919384338-8288-0/minicluster-data/ts-1/data/info.pb
I20260808 16:52:21.554327 15470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:21.552581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:45953 (local address 127.8.24.62:37541)
0808 16:52:21.552624 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:21.552626 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:21.552820 (+ 194us) server_negotiation.cc:407] Connection header received
0808 16:52:21.553190 (+ 370us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:21.553191 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:21.553196 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:21.553242 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:21.554151 (+ 909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:21.554152 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:21.554152 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 16:52:21.554156 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:21.554171 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 16:52:21.554182 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:21.554224 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:21.554265 (+ 41us) server_negotiation.cc:299] Negotiation successful
0808 16:52:21.554276 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 16:52:21.554519 15922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:21.552560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:37541 (local address 127.8.24.2:45953)
0808 16:52:21.552721 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:21.552754 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:21.553118 (+ 364us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:21.553293 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:21.553296 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:21.553303 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:21.554032 (+ 729us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 16:52:21.554032 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:21.554056 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 16:52:21.554080 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:52:21.554222 (+ 142us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:52:21.554224 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:52:21.554226 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 16:52:21.554270 (+ 44us) client_negotiation.cc:240] Negotiation successful
0808 16:52:21.554312 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":61,"threads_started":1}
I20260808 16:52:21.554968 15919 heartbeater.cc:347] Connected to a master server at 127.8.24.62:37541
I20260808 16:52:21.555045 15919 heartbeater.cc:464] Registering TS with master...
I20260808 16:52:21.555199 15919 heartbeater.cc:511] Master 127.8.24.62:37541 requested a full tablet report, sending...
I20260808 16:52:21.555202 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 15698
I20260808 16:52:21.555284 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786207919384338-8288-0/minicluster-data/ts-1/wal/instance
I20260808 16:52:21.555474 8288 external_mini_cluster.cc:1428] Setting key 0af92ae9c886d8297079215b7c05d873
I20260808 16:52:21.555527 15414 ts_manager.cc:194] Registered new tserver with Master: 2d2bd967f0f64fc3a9a5853787f772e0 (127.8.24.2:37745)
I20260808 16:52:21.556305 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786207919384338-8288-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786207919384338-8288-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786207919384338-8288-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786207919384338-8288-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.3:0
--local_ip_for_outbound_sockets=127.8.24.3
--webserver_interface=127.8.24.3
--webserver_port=0
--tserver_master_addrs=127.8.24.62:37541
--builtin_ntp_servers=127.8.24.20:41463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 16:52:21.634097 15924 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:52:21.634282 15924 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:52:21.634302 15924 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:52:21.635425 15924 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:52:21.635462 15924 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:52:21.635486 15924 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.3
I20260808 16:52:21.636821 15924 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:41463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786207919384338-8288-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786207919384338-8288-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786207919384338-8288-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.24.3
--webserver_port=0
--tserver_master_addrs=127.8.24.62:37541
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15924
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.3
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786207919384338-8288-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:52:21.637017 15924 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:52:21.637279 15924 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:52:21.641772 15933 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:21.641760 15936 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:21.641763 15934 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:21.641947 15924 server_base.cc:1034] running on GCE node
I20260808 16:52:21.642149 15924 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:52:21.642396 15924 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:52:21.643532 15924 hybrid_clock.cc:648] HybridClock initialized: now 1786207941643517 us; error 28 us; skew 500 ppm
I20260808 16:52:21.644598 15924 webserver.cc:460] Webserver started at http://127.8.24.3:35303/ using document root <none> and password file <none>
I20260808 16:52:21.644819 15924 fs_manager.cc:359] Metadata directory not provided
I20260808 16:52:21.644865 15924 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:52:21.644989 15924 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:52:21.645982 15924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786207919384338-8288-0/minicluster-data/ts-2/data/instance:
uuid: "9b147f4593bc4c709990991e2a3d432c"
format_stamp: "Formatted at 2026-08-08 16:52:21 on dist-test-slave-6zbq"
server_key: "f59b581ba0c49667c126b92692c76d5f"
server_key_iv: "8f1d102cdf1bda7d76f7e0cb5d279acc"
server_key_version: "encryptionkey@0"
I20260808 16:52:21.646303 15924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786207919384338-8288-0/minicluster-data/ts-2/wal/instance:
uuid: "9b147f4593bc4c709990991e2a3d432c"
format_stamp: "Formatted at 2026-08-08 16:52:21 on dist-test-slave-6zbq"
server_key: "f59b581ba0c49667c126b92692c76d5f"
server_key_iv: "8f1d102cdf1bda7d76f7e0cb5d279acc"
server_key_version: "encryptionkey@0"
I20260808 16:52:21.647462 15924 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 16:52:21.648057 15948 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:21.648193 15924 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 16:52:21.648272 15924 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786207919384338-8288-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786207919384338-8288-0/minicluster-data/ts-2/wal
uuid: "9b147f4593bc4c709990991e2a3d432c"
format_stamp: "Formatted at 2026-08-08 16:52:21 on dist-test-slave-6zbq"
server_key: "f59b581ba0c49667c126b92692c76d5f"
server_key_iv: "8f1d102cdf1bda7d76f7e0cb5d279acc"
server_key_version: "encryptionkey@0"
I20260808 16:52:21.648336 15924 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786207919384338-8288-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786207919384338-8288-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786207919384338-8288-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:52:21.651413 15924 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:52:21.651629 15924 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:52:21.651741 15924 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:52:21.651919 15924 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:52:21.652189 15924 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:52:21.652220 15924 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:21.652244 15924 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:52:21.652268 15924 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:21.657390 15924 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.3:46271
I20260808 16:52:21.657426 16130 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.3:46271 every 8 connection(s)
I20260808 16:52:21.657685 15924 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786207919384338-8288-0/minicluster-data/ts-2/data/info.pb
I20260808 16:52:21.660229 15470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:21.658257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:56909 (local address 127.8.24.62:37541)
0808 16:52:21.658305 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:21.658307 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:21.658549 (+ 242us) server_negotiation.cc:407] Connection header received
0808 16:52:21.658961 (+ 412us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:21.658962 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:21.658970 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:21.659021 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:21.659987 (+ 966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:21.659988 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:21.659989 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 16:52:21.659993 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:21.660011 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 16:52:21.660015 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:21.660064 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:21.660145 (+ 81us) server_negotiation.cc:299] Negotiation successful
0808 16:52:21.660161 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 16:52:21.660387 16136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:21.658265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:37541 (local address 127.8.24.3:56909)
0808 16:52:21.658450 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:21.658478 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:21.658872 (+ 394us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:21.659081 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:21.659085 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:21.659094 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:21.659851 (+ 757us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 16:52:21.659852 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:21.659877 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 16:52:21.659901 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:52:21.660067 (+ 166us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:52:21.660071 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:52:21.660075 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 16:52:21.660151 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 16:52:21.660207 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":95,"threads_started":1}
I20260808 16:52:21.660775 16133 heartbeater.cc:347] Connected to a master server at 127.8.24.62:37541
I20260808 16:52:21.660851 16133 heartbeater.cc:464] Registering TS with master...
I20260808 16:52:21.660830 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 15924
I20260808 16:52:21.660918 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786207919384338-8288-0/minicluster-data/ts-2/wal/instance
I20260808 16:52:21.660983 16133 heartbeater.cc:511] Master 127.8.24.62:37541 requested a full tablet report, sending...
I20260808 16:52:21.661108 8288 external_mini_cluster.cc:1428] Setting key dfb172318aeebc4deb0c930cb8ed4775
I20260808 16:52:21.661338 15414 ts_manager.cc:194] Registered new tserver with Master: 9b147f4593bc4c709990991e2a3d432c (127.8.24.3:46271)
I20260808 16:52:21.661964 8288 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 16:52:21.666517 15470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:21.663169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53614 (local address 127.8.24.62:37541)
0808 16:52:21.663235 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:21.663237 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:21.663332 (+ 95us) server_negotiation.cc:407] Connection header received
0808 16:52:21.663377 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:21.663377 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:21.663381 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:21.663412 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:21.666300 (+ 2888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:21.666302 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:21.666302 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 16:52:21.666305 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:21.666320 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 16:52:21.666323 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:21.666407 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:21.666446 (+ 39us) server_negotiation.cc:299] Negotiation successful
0808 16:52:21.666457 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 16:52:21.667922 15414 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53614:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 16:52:21.669327 15414 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 16:52:21.672636 16144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:21.671107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.3:46271 (local address 127.0.0.1:49686)
0808 16:52:21.671287 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:21.671322 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:21.671365 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:21.671830 (+ 465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:21.671831 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:21.671833 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:21.672284 (+ 451us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 16:52:21.672284 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:21.672289 (+ 5us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 16:52:21.672297 (+ 8us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:52:21.672486 (+ 189us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:52:21.672493 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:52:21.672495 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 16:52:21.672551 (+ 56us) client_negotiation.cc:240] Negotiation successful
0808 16:52:21.672564 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":59,"threads_started":1}
I20260808 16:52:21.672654 16145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:21.671170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49686 (local address 127.8.24.3:46271)
0808 16:52:21.671464 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:21.671467 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:21.671475 (+ 8us) server_negotiation.cc:407] Connection header received
0808 16:52:21.671495 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:21.671497 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:21.671505 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:21.671565 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:21.672356 (+ 791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:21.672363 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:21.672364 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 16:52:21.672368 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:21.672403 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 16:52:21.672423 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:21.672485 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:21.672567 (+ 82us) server_negotiation.cc:299] Negotiation successful
0808 16:52:21.672581 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":261,"thread_start_us":68,"threads_started":1}
I20260808 16:52:21.672793 16148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:21.671147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:45505 (local address 127.0.0.1:48464)
0808 16:52:21.671431 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:21.671441 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:21.671477 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:21.671722 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:21.671728 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:21.671734 (+ 6us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:21.672191 (+ 457us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 16:52:21.672192 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:21.672207 (+ 15us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 16:52:21.672227 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:52:21.672675 (+ 448us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:52:21.672676 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:52:21.672677 (+ 1us) client_negotiation.cc:769] Sending connection context
0808 16:52:21.672708 (+ 31us) client_negotiation.cc:240] Negotiation successful
0808 16:52:21.672721 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":87,"threads_started":1}
I20260808 16:52:21.672794 16143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:21.671303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.2:37745 (local address 127.0.0.1:54976)
0808 16:52:21.671605 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:21.671614 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:21.671672 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:21.671934 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:21.671935 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:21.671938 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:21.672437 (+ 499us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 16:52:21.672437 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:21.672440 (+ 3us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 16:52:21.672447 (+ 7us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:52:21.672670 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:52:21.672671 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:52:21.672671 (+ 0us) client_negotiation.cc:769] Sending connection context
0808 16:52:21.672715 (+ 44us) client_negotiation.cc:240] Negotiation successful
0808 16:52:21.672727 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":73,"threads_started":1}
I20260808 16:52:21.672912 16149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:21.671391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54976 (local address 127.8.24.2:37745)
0808 16:52:21.671585 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:21.671588 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:21.671664 (+ 76us) server_negotiation.cc:407] Connection header received
0808 16:52:21.671745 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:21.671752 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:21.671761 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:21.671835 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:21.672522 (+ 687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:21.672528 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:21.672528 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 16:52:21.672533 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:21.672567 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 16:52:21.672594 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:21.672644 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:21.672822 (+ 178us) server_negotiation.cc:299] Negotiation successful
0808 16:52:21.672836 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":76,"threads_started":1}
I20260808 16:52:21.672991 16150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:21.671446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48464 (local address 127.8.24.1:45505)
0808 16:52:21.671583 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:21.671587 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:21.671598 (+ 11us) server_negotiation.cc:407] Connection header received
0808 16:52:21.671625 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:21.671629 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:21.671638 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:21.671704 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:21.672549 (+ 845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:21.672552 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:21.672553 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 16:52:21.672556 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:21.672583 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 16:52:21.672597 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:21.672874 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:21.672897 (+ 23us) server_negotiation.cc:299] Negotiation successful
0808 16:52:21.672913 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":71,"threads_started":1}
I20260808 16:52:21.673535 16022 tablet_service.cc:1467] Processing CreateTablet for tablet 6812b4e75b2b444cb3646daf054fc448 (DEFAULT_TABLE table=test-table [id=19b204f4cbd34ceda8a95fb29929c734]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:52:21.673619 15592 tablet_service.cc:1467] Processing CreateTablet for tablet 6812b4e75b2b444cb3646daf054fc448 (DEFAULT_TABLE table=test-table [id=19b204f4cbd34ceda8a95fb29929c734]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:52:21.673812 16022 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6812b4e75b2b444cb3646daf054fc448. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:52:21.673954 15592 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6812b4e75b2b444cb3646daf054fc448. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:52:21.676180 16156 tablet_bootstrap.cc:492] T 6812b4e75b2b444cb3646daf054fc448 P 9b147f4593bc4c709990991e2a3d432c: Bootstrap starting.
I20260808 16:52:21.676425 15805 tablet_service.cc:1467] Processing CreateTablet for tablet 6812b4e75b2b444cb3646daf054fc448 (DEFAULT_TABLE table=test-table [id=19b204f4cbd34ceda8a95fb29929c734]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:52:21.676745 15805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6812b4e75b2b444cb3646daf054fc448. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:52:21.676915 16156 tablet_bootstrap.cc:654] T 6812b4e75b2b444cb3646daf054fc448 P 9b147f4593bc4c709990991e2a3d432c: Neither blocks nor log segments found. Creating new log.
I20260808 16:52:21.677208 16156 log.cc:824] T 6812b4e75b2b444cb3646daf054fc448 P 9b147f4593bc4c709990991e2a3d432c: Log is configured to *not* fsync() on all Append() calls
I20260808 16:52:21.677908 16156 tablet_bootstrap.cc:492] T 6812b4e75b2b444cb3646daf054fc448 P 9b147f4593bc4c709990991e2a3d432c: No bootstrap required, opened a new log
I20260808 16:52:21.677980 16156 ts_tablet_manager.cc:1397] T 6812b4e75b2b444cb3646daf054fc448 P 9b147f4593bc4c709990991e2a3d432c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 16:52:21.678488 16157 tablet_bootstrap.cc:492] T 6812b4e75b2b444cb3646daf054fc448 P 3de19bfd237d4931a04aad8b15204456: Bootstrap starting.
I20260808 16:52:21.679028 16157 tablet_bootstrap.cc:654] T 6812b4e75b2b444cb3646daf054fc448 P 3de19bfd237d4931a04aad8b15204456: Neither blocks nor log segments found. Creating new log.
I20260808 16:52:21.679333 16157 log.cc:824] T 6812b4e75b2b444cb3646daf054fc448 P 3de19bfd237d4931a04aad8b15204456: Log is configured to *not* fsync() on all Append() calls
I20260808 16:52:21.679979 16157 tablet_bootstrap.cc:492] T 6812b4e75b2b444cb3646daf054fc448 P 3de19bfd237d4931a04aad8b15204456: No bootstrap required, opened a new log
I20260808 16:52:21.680038 16157 ts_tablet_manager.cc:1397] T 6812b4e75b2b444cb3646daf054fc448 P 3de19bfd237d4931a04aad8b15204456: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 16:52:21.680316 16156 raft_consensus.cc:348] T 6812b4e75b2b444cb3646daf054fc448 P 9b147f4593bc4c709990991e2a3d432c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b147f4593bc4c709990991e2a3d432c" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 46271 } } peers { permanent_uuid: "2d2bd967f0f64fc3a9a5853787f772e0" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 37745 } } peers { permanent_uuid: "3de19bfd237d4931a04aad8b15204456" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 45505 } }
I20260808 16:52:21.680435 16156 raft_consensus.cc:374] T 6812b4e75b2b444cb3646daf054fc448 P 9b147f4593bc4c709990991e2a3d432c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:52:21.680455 16156 raft_consensus.cc:729] T 6812b4e75b2b444cb3646daf054fc448 P 9b147f4593bc4c709990991e2a3d432c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b147f4593bc4c709990991e2a3d432c, State: Initialized, Role: FOLLOWER
I20260808 16:52:21.680560 16156 consensus_queue.cc:260] T 6812b4e75b2b444cb3646daf054fc448 P 9b147f4593bc4c709990991e2a3d432c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b147f4593bc4c709990991e2a3d432c" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 46271 } } peers { permanent_uuid: "2d2bd967f0f64fc3a9a5853787f772e0" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 37745 } } peers { permanent_uuid: "3de19bfd237d4931a04aad8b15204456" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 45505 } }
I20260808 16:52:21.680801 16156 ts_tablet_manager.cc:1428] T 6812b4e75b2b444cb3646daf054fc448 P 9b147f4593bc4c709990991e2a3d432c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 16:52:21.680812 16133 heartbeater.cc:503] Master 127.8.24.62:37541 was elected leader, sending a full tablet report...
I20260808 16:52:21.682212 16157 raft_consensus.cc:348] T 6812b4e75b2b444cb3646daf054fc448 P 3de19bfd237d4931a04aad8b15204456 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b147f4593bc4c709990991e2a3d432c" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 46271 } } peers { permanent_uuid: "2d2bd967f0f64fc3a9a5853787f772e0" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 37745 } } peers { permanent_uuid: "3de19bfd237d4931a04aad8b15204456" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 45505 } }
I20260808 16:52:21.682394 16157 raft_consensus.cc:374] T 6812b4e75b2b444cb3646daf054fc448 P 3de19bfd237d4931a04aad8b15204456 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:52:21.682441 16157 raft_consensus.cc:729] T 6812b4e75b2b444cb3646daf054fc448 P 3de19bfd237d4931a04aad8b15204456 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3de19bfd237d4931a04aad8b15204456, State: Initialized, Role: FOLLOWER
I20260808 16:52:21.682534 16161 tablet_bootstrap.cc:492] T 6812b4e75b2b444cb3646daf054fc448 P 2d2bd967f0f64fc3a9a5853787f772e0: Bootstrap starting.
I20260808 16:52:21.682554 16157 consensus_queue.cc:260] T 6812b4e75b2b444cb3646daf054fc448 P 3de19bfd237d4931a04aad8b15204456 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b147f4593bc4c709990991e2a3d432c" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 46271 } } peers { permanent_uuid: "2d2bd967f0f64fc3a9a5853787f772e0" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 37745 } } peers { permanent_uuid: "3de19bfd237d4931a04aad8b15204456" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 45505 } }
I20260808 16:52:21.682807 16157 ts_tablet_manager.cc:1428] T 6812b4e75b2b444cb3646daf054fc448 P 3de19bfd237d4931a04aad8b15204456: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 16:52:21.682862 15692 heartbeater.cc:503] Master 127.8.24.62:37541 was elected leader, sending a full tablet report...
I20260808 16:52:21.683166 16161 tablet_bootstrap.cc:654] T 6812b4e75b2b444cb3646daf054fc448 P 2d2bd967f0f64fc3a9a5853787f772e0: Neither blocks nor log segments found. Creating new log.
I20260808 16:52:21.683465 16161 log.cc:824] T 6812b4e75b2b444cb3646daf054fc448 P 2d2bd967f0f64fc3a9a5853787f772e0: Log is configured to *not* fsync() on all Append() calls
I20260808 16:52:21.684149 16161 tablet_bootstrap.cc:492] T 6812b4e75b2b444cb3646daf054fc448 P 2d2bd967f0f64fc3a9a5853787f772e0: No bootstrap required, opened a new log
I20260808 16:52:21.684201 16161 ts_tablet_manager.cc:1397] T 6812b4e75b2b444cb3646daf054fc448 P 2d2bd967f0f64fc3a9a5853787f772e0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 16:52:21.685823 16161 raft_consensus.cc:348] T 6812b4e75b2b444cb3646daf054fc448 P 2d2bd967f0f64fc3a9a5853787f772e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b147f4593bc4c709990991e2a3d432c" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 46271 } } peers { permanent_uuid: "2d2bd967f0f64fc3a9a5853787f772e0" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 37745 } } peers { permanent_uuid: "3de19bfd237d4931a04aad8b15204456" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 45505 } }
I20260808 16:52:21.685936 16161 raft_consensus.cc:374] T 6812b4e75b2b444cb3646daf054fc448 P 2d2bd967f0f64fc3a9a5853787f772e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:52:21.685956 16161 raft_consensus.cc:729] T 6812b4e75b2b444cb3646daf054fc448 P 2d2bd967f0f64fc3a9a5853787f772e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d2bd967f0f64fc3a9a5853787f772e0, State: Initialized, Role: FOLLOWER
I20260808 16:52:21.686028 16161 consensus_queue.cc:260] T 6812b4e75b2b444cb3646daf054fc448 P 2d2bd967f0f64fc3a9a5853787f772e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b147f4593bc4c709990991e2a3d432c" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 46271 } } peers { permanent_uuid: "2d2bd967f0f64fc3a9a5853787f772e0" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 37745 } } peers { permanent_uuid: "3de19bfd237d4931a04aad8b15204456" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 45505 } }
I20260808 16:52:21.686219 16161 ts_tablet_manager.cc:1428] T 6812b4e75b2b444cb3646daf054fc448 P 2d2bd967f0f64fc3a9a5853787f772e0: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 16:52:21.686283 15919 heartbeater.cc:503] Master 127.8.24.62:37541 was elected leader, sending a full tablet report...
W20260808 16:52:21.697541 15694 tablet.cc:2369] T 6812b4e75b2b444cb3646daf054fc448 P 3de19bfd237d4931a04aad8b15204456: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 16:52:21.799289 16163 raft_consensus.cc:482] T 6812b4e75b2b444cb3646daf054fc448 P 9b147f4593bc4c709990991e2a3d432c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 16:52:21.799408 16163 raft_consensus.cc:504] T 6812b4e75b2b444cb3646daf054fc448 P 9b147f4593bc4c709990991e2a3d432c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b147f4593bc4c709990991e2a3d432c" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 46271 } } peers { permanent_uuid: "2d2bd967f0f64fc3a9a5853787f772e0" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 37745 } } peers { permanent_uuid: "3de19bfd237d4931a04aad8b15204456" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 45505 } }
I20260808 16:52:21.799703 16163 leader_election.cc:288] T 6812b4e75b2b444cb3646daf054fc448 P 9b147f4593bc4c709990991e2a3d432c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2d2bd967f0f64fc3a9a5853787f772e0 (127.8.24.2:37745), 3de19bfd237d4931a04aad8b15204456 (127.8.24.1:45505)
I20260808 16:52:21.800827 16136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:21.799869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.2:37745 (local address 127.8.24.3:60517)
0808 16:52:21.799943 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:21.799953 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:21.799990 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:21.800130 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:21.800131 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:21.800133 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:21.800523 (+ 390us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 16:52:21.800523 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:21.800531 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 16:52:21.800544 (+ 13us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:52:21.800707 (+ 163us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:52:21.800708 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:52:21.800709 (+ 1us) client_negotiation.cc:769] Sending connection context
0808 16:52:21.800747 (+ 38us) client_negotiation.cc:240] Negotiation successful
0808 16:52:21.800759 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260808 16:52:21.800848 16149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:21.799926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:60517 (local address 127.8.24.2:37745)
0808 16:52:21.799970 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:21.799972 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:21.799983 (+ 11us) server_negotiation.cc:407] Connection header received
0808 16:52:21.800045 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:21.800046 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:21.800050 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:21.800116 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:21.800624 (+ 508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:21.800625 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:21.800626 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 16:52:21.800630 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:21.800646 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 16:52:21.800650 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:21.800703 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:21.800754 (+ 51us) server_negotiation.cc:299] Negotiation successful
0808 16:52:21.800768 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 16:52:21.800908 16150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:21.799949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:53121 (local address 127.8.24.1:45505)
0808 16:52:21.799986 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:21.799988 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:21.800077 (+ 89us) server_negotiation.cc:407] Connection header received
0808 16:52:21.800115 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:21.800115 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:21.800120 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:21.800167 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:21.800715 (+ 548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:21.800717 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:21.800717 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 16:52:21.800722 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:21.800737 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 16:52:21.800742 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:21.800794 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:21.800839 (+ 45us) server_negotiation.cc:299] Negotiation successful
0808 16:52:21.800851 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 16:52:21.800918 16168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:21.799869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:45505 (local address 127.8.24.3:53121)
0808 16:52:21.800017 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:21.800025 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:21.800068 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:21.800185 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:21.800186 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:21.800188 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:21.800628 (+ 440us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 16:52:21.800628 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:21.800634 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 16:52:21.800643 (+ 9us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:52:21.800797 (+ 154us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:52:21.800798 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:52:21.800798 (+ 0us) client_negotiation.cc:769] Sending connection context
0808 16:52:21.800839 (+ 41us) client_negotiation.cc:240] Negotiation successful
0808 16:52:21.800851 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"mutex_wait_us":12,"thread_start_us":83,"threads_started":1}
I20260808 16:52:21.801134 15843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6812b4e75b2b444cb3646daf054fc448" candidate_uuid: "9b147f4593bc4c709990991e2a3d432c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d2bd967f0f64fc3a9a5853787f772e0" is_pre_election: true
I20260808 16:52:21.801198 15629 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6812b4e75b2b444cb3646daf054fc448" candidate_uuid: "9b147f4593bc4c709990991e2a3d432c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3de19bfd237d4931a04aad8b15204456" is_pre_election: true
I20260808 16:52:21.801299 15843 raft_consensus.cc:2454] T 6812b4e75b2b444cb3646daf054fc448 P 2d2bd967f0f64fc3a9a5853787f772e0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9b147f4593bc4c709990991e2a3d432c in term 0.
I20260808 16:52:21.801323 15629 raft_consensus.cc:2454] T 6812b4e75b2b444cb3646daf054fc448 P 3de19bfd237d4931a04aad8b15204456 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9b147f4593bc4c709990991e2a3d432c in term 0.
I20260808 16:52:21.801542 15954 leader_election.cc:302] T 6812b4e75b2b444cb3646daf054fc448 P 9b147f4593bc4c709990991e2a3d432c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3de19bfd237d4931a04aad8b15204456, 9b147f4593bc4c709990991e2a3d432c; no voters:
I20260808 16:52:21.801685 16163 raft_consensus.cc:2781] T 6812b4e75b2b444cb3646daf054fc448 P 9b147f4593bc4c709990991e2a3d432c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 16:52:21.801753 16163 raft_consensus.cc:482] T 6812b4e75b2b444cb3646daf054fc448 P 9b147f4593bc4c709990991e2a3d432c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 16:52:21.801769 16163 raft_consensus.cc:3037] T 6812b4e75b2b444cb3646daf054fc448 P 9b147f4593bc4c709990991e2a3d432c [term 0 FOLLOWER]: Advancing to term 1
W20260808 16:52:21.802471 15920 tablet.cc:2369] T 6812b4e75b2b444cb3646daf054fc448 P 2d2bd967f0f64fc3a9a5853787f772e0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 16:52:21.802528 16163 raft_consensus.cc:504] T 6812b4e75b2b444cb3646daf054fc448 P 9b147f4593bc4c709990991e2a3d432c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b147f4593bc4c709990991e2a3d432c" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 46271 } } peers { permanent_uuid: "2d2bd967f0f64fc3a9a5853787f772e0" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 37745 } } peers { permanent_uuid: "3de19bfd237d4931a04aad8b15204456" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 45505 } }
I20260808 16:52:21.802659 16163 leader_election.cc:288] T 6812b4e75b2b444cb3646daf054fc448 P 9b147f4593bc4c709990991e2a3d432c [CANDIDATE]: Term 1 election: Requested vote from peers 2d2bd967f0f64fc3a9a5853787f772e0 (127.8.24.2:37745), 3de19bfd237d4931a04aad8b15204456 (127.8.24.1:45505)
I20260808 16:52:21.802827 15629 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6812b4e75b2b444cb3646daf054fc448" candidate_uuid: "9b147f4593bc4c709990991e2a3d432c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3de19bfd237d4931a04aad8b15204456"
I20260808 16:52:21.802845 15843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6812b4e75b2b444cb3646daf054fc448" candidate_uuid: "9b147f4593bc4c709990991e2a3d432c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d2bd967f0f64fc3a9a5853787f772e0"
I20260808 16:52:21.802893 15629 raft_consensus.cc:3037] T 6812b4e75b2b444cb3646daf054fc448 P 3de19bfd237d4931a04aad8b15204456 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:52:21.802911 15843 raft_consensus.cc:3037] T 6812b4e75b2b444cb3646daf054fc448 P 2d2bd967f0f64fc3a9a5853787f772e0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:52:21.803555 15629 raft_consensus.cc:2454] T 6812b4e75b2b444cb3646daf054fc448 P 3de19bfd237d4931a04aad8b15204456 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b147f4593bc4c709990991e2a3d432c in term 1.
I20260808 16:52:21.803599 15843 raft_consensus.cc:2454] T 6812b4e75b2b444cb3646daf054fc448 P 2d2bd967f0f64fc3a9a5853787f772e0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b147f4593bc4c709990991e2a3d432c in term 1.
I20260808 16:52:21.803738 15954 leader_election.cc:302] T 6812b4e75b2b444cb3646daf054fc448 P 9b147f4593bc4c709990991e2a3d432c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3de19bfd237d4931a04aad8b15204456, 9b147f4593bc4c709990991e2a3d432c; no voters:
I20260808 16:52:21.803824 16163 raft_consensus.cc:2781] T 6812b4e75b2b444cb3646daf054fc448 P 9b147f4593bc4c709990991e2a3d432c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:52:21.803961 16163 raft_consensus.cc:686] T 6812b4e75b2b444cb3646daf054fc448 P 9b147f4593bc4c709990991e2a3d432c [term 1 LEADER]: Becoming Leader. State: Replica: 9b147f4593bc4c709990991e2a3d432c, State: Running, Role: LEADER
I20260808 16:52:21.804050 16163 consensus_queue.cc:237] T 6812b4e75b2b444cb3646daf054fc448 P 9b147f4593bc4c709990991e2a3d432c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b147f4593bc4c709990991e2a3d432c" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 46271 } } peers { permanent_uuid: "2d2bd967f0f64fc3a9a5853787f772e0" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 37745 } } peers { permanent_uuid: "3de19bfd237d4931a04aad8b15204456" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 45505 } }
I20260808 16:52:21.804791 15414 catalog_manager.cc:5495] T 6812b4e75b2b444cb3646daf054fc448 P 9b147f4593bc4c709990991e2a3d432c reported cstate change: term changed from 0 to 1, leader changed from <none> to 9b147f4593bc4c709990991e2a3d432c (127.8.24.3). New cstate: current_term: 1 leader_uuid: "9b147f4593bc4c709990991e2a3d432c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b147f4593bc4c709990991e2a3d432c" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 46271 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2d2bd967f0f64fc3a9a5853787f772e0" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 37745 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3de19bfd237d4931a04aad8b15204456" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 45505 } health_report { overall_health: UNKNOWN } } }
I20260808 16:52:21.815502 16145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:21.814682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49698 (local address 127.8.24.3:46271)
0808 16:52:21.814735 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:21.814737 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:21.814745 (+ 8us) server_negotiation.cc:407] Connection header received
0808 16:52:21.814789 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:21.814790 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:21.814800 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:21.814850 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:21.815314 (+ 464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:21.815315 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:21.815316 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 16:52:21.815320 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:21.815333 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 16:52:21.815337 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:21.815384 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:21.815433 (+ 49us) server_negotiation.cc:299] Negotiation successful
0808 16:52:21.815444 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 16:52:21.816910 15629 raft_consensus.cc:1261] T 6812b4e75b2b444cb3646daf054fc448 P 3de19bfd237d4931a04aad8b15204456 [term 1 FOLLOWER]: Refusing update from remote peer 9b147f4593bc4c709990991e2a3d432c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:52:21.816922 15843 raft_consensus.cc:1261] T 6812b4e75b2b444cb3646daf054fc448 P 2d2bd967f0f64fc3a9a5853787f772e0 [term 1 FOLLOWER]: Refusing update from remote peer 9b147f4593bc4c709990991e2a3d432c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:52:21.817191 16170 consensus_queue.cc:1035] T 6812b4e75b2b444cb3646daf054fc448 P 9b147f4593bc4c709990991e2a3d432c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3de19bfd237d4931a04aad8b15204456" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 45505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:52:21.817306 16163 consensus_queue.cc:1035] T 6812b4e75b2b444cb3646daf054fc448 P 9b147f4593bc4c709990991e2a3d432c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d2bd967f0f64fc3a9a5853787f772e0" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 37745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:52:21.819444 16173 mvcc.cc:205] Tried to move back new op lower bound from 7316307729680056320 to 7316307729629945856. Current Snapshot: MvccSnapshot[applied={T|T < 7316307729680056320}]
I20260808 16:52:21.819533 16178 mvcc.cc:205] Tried to move back new op lower bound from 7316307729680056320 to 7316307729629945856. Current Snapshot: MvccSnapshot[applied={T|T < 7316307729680056320}]
I20260808 16:52:21.819623 16176 mvcc.cc:205] Tried to move back new op lower bound from 7316307729680056320 to 7316307729629945856. Current Snapshot: MvccSnapshot[applied={T|T < 7316307729680056320}]
I20260808 16:52:21.821568 15412 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:53614:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 16:52:21.821640 15412 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:53614:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 16:52:21.822300 15412 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1523edb278ab40ec961207d0e615810f: Sending DeleteTablet for 3 replicas of tablet 6812b4e75b2b444cb3646daf054fc448
I20260808 16:52:21.822651 15805 tablet_service.cc:1514] Processing DeleteTablet for tablet 6812b4e75b2b444cb3646daf054fc448 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:52:21 UTC) from {username='slave'} at 127.0.0.1:54976
I20260808 16:52:21.822649 15592 tablet_service.cc:1514] Processing DeleteTablet for tablet 6812b4e75b2b444cb3646daf054fc448 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:52:21 UTC) from {username='slave'} at 127.0.0.1:48464
I20260808 16:52:21.822695 16022 tablet_service.cc:1514] Processing DeleteTablet for tablet 6812b4e75b2b444cb3646daf054fc448 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:52:21 UTC) from {username='slave'} at 127.0.0.1:49686
I20260808 16:52:21.822935 16189 tablet_replica.cc:330] T 6812b4e75b2b444cb3646daf054fc448 P 3de19bfd237d4931a04aad8b15204456: stopping tablet replica
I20260808 16:52:21.823028 16189 raft_consensus.cc:2229] T 6812b4e75b2b444cb3646daf054fc448 P 3de19bfd237d4931a04aad8b15204456 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 16:52:21.823127 16189 raft_consensus.cc:2258] T 6812b4e75b2b444cb3646daf054fc448 P 3de19bfd237d4931a04aad8b15204456 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:52:21.823117 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 15501
I20260808 16:52:21.823280 16187 tablet_replica.cc:330] T 6812b4e75b2b444cb3646daf054fc448 P 2d2bd967f0f64fc3a9a5853787f772e0: stopping tablet replica
I20260808 16:52:21.823356 16187 raft_consensus.cc:2229] T 6812b4e75b2b444cb3646daf054fc448 P 2d2bd967f0f64fc3a9a5853787f772e0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 16:52:21.823424 16187 raft_consensus.cc:2258] T 6812b4e75b2b444cb3646daf054fc448 P 2d2bd967f0f64fc3a9a5853787f772e0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:52:21.823535 16196 tablet_replica.cc:330] T 6812b4e75b2b444cb3646daf054fc448 P 9b147f4593bc4c709990991e2a3d432c: stopping tablet replica
I20260808 16:52:21.823598 16196 raft_consensus.cc:2229] T 6812b4e75b2b444cb3646daf054fc448 P 9b147f4593bc4c709990991e2a3d432c [term 1 LEADER]: Raft consensus shutting down.
I20260808 16:52:21.823666 16196 raft_consensus.cc:2258] T 6812b4e75b2b444cb3646daf054fc448 P 9b147f4593bc4c709990991e2a3d432c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:52:21.826081 16196 ts_tablet_manager.cc:1905] T 6812b4e75b2b444cb3646daf054fc448 P 9b147f4593bc4c709990991e2a3d432c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:52:21.827627 16196 ts_tablet_manager.cc:1918] T 6812b4e75b2b444cb3646daf054fc448 P 9b147f4593bc4c709990991e2a3d432c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 16:52:21.827689 16196 log.cc:1196] T 6812b4e75b2b444cb3646daf054fc448 P 9b147f4593bc4c709990991e2a3d432c: Deleting WAL directory at /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786207919384338-8288-0/minicluster-data/ts-2/wal/wals/6812b4e75b2b444cb3646daf054fc448
I20260808 16:52:21.827904 16196 ts_tablet_manager.cc:1939] T 6812b4e75b2b444cb3646daf054fc448 P 9b147f4593bc4c709990991e2a3d432c: Deleting consensus metadata
I20260808 16:52:21.843731 15392 catalog_manager.cc:4841] TS 9b147f4593bc4c709990991e2a3d432c (127.8.24.3:46271): tablet 6812b4e75b2b444cb3646daf054fc448 (table test-table [id=19b204f4cbd34ceda8a95fb29929c734]) successfully deleted
I20260808 16:52:21.844920 16187 ts_tablet_manager.cc:1905] T 6812b4e75b2b444cb3646daf054fc448 P 2d2bd967f0f64fc3a9a5853787f772e0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:52:21.846637 16187 ts_tablet_manager.cc:1918] T 6812b4e75b2b444cb3646daf054fc448 P 2d2bd967f0f64fc3a9a5853787f772e0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 16:52:21.846711 16187 log.cc:1196] T 6812b4e75b2b444cb3646daf054fc448 P 2d2bd967f0f64fc3a9a5853787f772e0: Deleting WAL directory at /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786207919384338-8288-0/minicluster-data/ts-1/wal/wals/6812b4e75b2b444cb3646daf054fc448
I20260808 16:52:21.846975 16187 ts_tablet_manager.cc:1939] T 6812b4e75b2b444cb3646daf054fc448 P 2d2bd967f0f64fc3a9a5853787f772e0: Deleting consensus metadata
I20260808 16:52:21.847404 15392 catalog_manager.cc:4841] TS 2d2bd967f0f64fc3a9a5853787f772e0 (127.8.24.2:37745): tablet 6812b4e75b2b444cb3646daf054fc448 (table test-table [id=19b204f4cbd34ceda8a95fb29929c734]) successfully deleted
W20260808 16:52:21.848946 15394 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.24.1:45505 (error 108)
I20260808 16:52:21.849141 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 15698
I20260808 16:52:21.850526 16144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:21.849283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:45505 (local address 127.0.0.1:48468)
0808 16:52:21.850443 (+ 1160us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:21.850495 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.1:45505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1135}
W20260808 16:52:21.852768 15394 catalog_manager.cc:4584] TS 3de19bfd237d4931a04aad8b15204456 (127.8.24.1:45505): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6812b4e75b2b444cb3646daf054fc448: Network error: Client connection negotiation failed: client connection to 127.8.24.1:45505: connect: Connection refused (error 111)
I20260808 16:52:21.859948 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 15924
I20260808 16:52:21.870393 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 15363
2026-08-08T16:52:21Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (722 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260808 16:52:21.891976 8288 mini_oidc.cc:152] Starting JWKS server
I20260808 16:52:21.893148 8288 webserver.cc:460] Webserver started at https://127.0.0.1:37839/ using document root <none> and password file <none>
I20260808 16:52:21.893240 8288 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 16:52:21.893404 8288 webserver.cc:460] Webserver started at http://127.0.0.1:37539/ using document root <none> and password file <none>
2026-08-08T16:52:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T16:52:21Z Disabled control of system clock
I20260808 16:52:21.900893 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786207919384338-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786207919384338-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786207919384338-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:37009
--webserver_interface=127.8.24.62
--webserver_port=0
--builtin_ntp_servers=127.8.24.20:34383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:37009
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37839/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786207919384338-8288-0/testchainca.cert with env {}
W20260808 16:52:21.978257 16211 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:52:21.978471 16211 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:52:21.978497 16211 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:52:21.979583 16211 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 16:52:21.979614 16211 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:52:21.979624 16211 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:52:21.979635 16211 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 16:52:21.979646 16211 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 16:52:21.979656 16211 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 16:52:21.979666 16211 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:37839/jwks.json
I20260808 16:52:21.981052 16211 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:34383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786207919384338-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786207919384338-8288-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:37009
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:37009
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37839/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.24.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786207919384338-8288-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16211
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786207919384338-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:52:21.981244 16211 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:52:21.981398 16211 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:52:21.985663 16224 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:21.985746 16211 server_base.cc:1034] running on GCE node
W20260808 16:52:21.985663 16220 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:21.985661 16221 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:21.986135 16211 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:52:21.986443 16211 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:52:21.987583 16211 hybrid_clock.cc:648] HybridClock initialized: now 1786207941987573 us; error 27 us; skew 500 ppm
I20260808 16:52:21.988643 16211 webserver.cc:460] Webserver started at http://127.8.24.62:45783/ using document root <none> and password file <none>
I20260808 16:52:21.988842 16211 fs_manager.cc:359] Metadata directory not provided
I20260808 16:52:21.988883 16211 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:52:21.988996 16211 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:52:21.989928 16211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786207919384338-8288-0/minicluster-data/master-0/data/instance:
uuid: "4141c4d302754d4b9c22b08ef5c4771f"
format_stamp: "Formatted at 2026-08-08 16:52:21 on dist-test-slave-6zbq"
server_key: "908b07db34587c19291c06c7903f289c"
server_key_iv: "1042c5a0f7b369ce6c017d699bb7abcc"
server_key_version: "encryptionkey@0"
I20260808 16:52:21.990239 16211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786207919384338-8288-0/minicluster-data/master-0/wal/instance:
uuid: "4141c4d302754d4b9c22b08ef5c4771f"
format_stamp: "Formatted at 2026-08-08 16:52:21 on dist-test-slave-6zbq"
server_key: "908b07db34587c19291c06c7903f289c"
server_key_iv: "1042c5a0f7b369ce6c017d699bb7abcc"
server_key_version: "encryptionkey@0"
I20260808 16:52:21.991400 16211 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 16:52:21.991998 16235 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:21.992141 16211 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:21.992229 16211 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786207919384338-8288-0/minicluster-data/master-0/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786207919384338-8288-0/minicluster-data/master-0/wal
uuid: "4141c4d302754d4b9c22b08ef5c4771f"
format_stamp: "Formatted at 2026-08-08 16:52:21 on dist-test-slave-6zbq"
server_key: "908b07db34587c19291c06c7903f289c"
server_key_iv: "1042c5a0f7b369ce6c017d699bb7abcc"
server_key_version: "encryptionkey@0"
I20260808 16:52:21.992292 16211 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786207919384338-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786207919384338-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786207919384338-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:52:22.033241 16211 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:52:22.033483 16211 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:52:22.033597 16211 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:52:22.037187 16211 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.62:37009
I20260808 16:52:22.037220 16317 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.62:37009 every 8 connection(s)
I20260808 16:52:22.037492 16211 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
I20260808 16:52:22.038420 16318 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:52:22.040421 16318 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4141c4d302754d4b9c22b08ef5c4771f: Bootstrap starting.
I20260808 16:52:22.040921 16318 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4141c4d302754d4b9c22b08ef5c4771f: Neither blocks nor log segments found. Creating new log.
I20260808 16:52:22.041159 16318 log.cc:824] T 00000000000000000000000000000000 P 4141c4d302754d4b9c22b08ef5c4771f: Log is configured to *not* fsync() on all Append() calls
I20260808 16:52:22.041723 16318 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4141c4d302754d4b9c22b08ef5c4771f: No bootstrap required, opened a new log
I20260808 16:52:22.043226 16318 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4141c4d302754d4b9c22b08ef5c4771f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4141c4d302754d4b9c22b08ef5c4771f" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 37009 } }
I20260808 16:52:22.043349 16318 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4141c4d302754d4b9c22b08ef5c4771f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:52:22.043368 16318 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4141c4d302754d4b9c22b08ef5c4771f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4141c4d302754d4b9c22b08ef5c4771f, State: Initialized, Role: FOLLOWER
I20260808 16:52:22.043473 16318 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4141c4d302754d4b9c22b08ef5c4771f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4141c4d302754d4b9c22b08ef5c4771f" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 37009 } }
I20260808 16:52:22.043545 16318 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4141c4d302754d4b9c22b08ef5c4771f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 16:52:22.043572 16318 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4141c4d302754d4b9c22b08ef5c4771f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 16:52:22.043604 16318 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4141c4d302754d4b9c22b08ef5c4771f [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:52:22.044029 16318 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4141c4d302754d4b9c22b08ef5c4771f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4141c4d302754d4b9c22b08ef5c4771f" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 37009 } }
I20260808 16:52:22.044107 16318 leader_election.cc:302] T 00000000000000000000000000000000 P 4141c4d302754d4b9c22b08ef5c4771f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4141c4d302754d4b9c22b08ef5c4771f; no voters:
I20260808 16:52:22.044224 16318 leader_election.cc:288] T 00000000000000000000000000000000 P 4141c4d302754d4b9c22b08ef5c4771f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 16:52:22.044277 16323 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4141c4d302754d4b9c22b08ef5c4771f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:52:22.044425 16318 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4141c4d302754d4b9c22b08ef5c4771f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 16:52:22.044432 16323 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4141c4d302754d4b9c22b08ef5c4771f [term 1 LEADER]: Becoming Leader. State: Replica: 4141c4d302754d4b9c22b08ef5c4771f, State: Running, Role: LEADER
I20260808 16:52:22.044569 16323 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4141c4d302754d4b9c22b08ef5c4771f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4141c4d302754d4b9c22b08ef5c4771f" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 37009 } }
I20260808 16:52:22.044888 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 16211
I20260808 16:52:22.044986 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786207919384338-8288-0/minicluster-data/master-0/wal/instance
I20260808 16:52:22.044950 16324 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4141c4d302754d4b9c22b08ef5c4771f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4141c4d302754d4b9c22b08ef5c4771f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4141c4d302754d4b9c22b08ef5c4771f" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 37009 } } }
I20260808 16:52:22.045022 16324 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4141c4d302754d4b9c22b08ef5c4771f [sys.catalog]: This master's current role is: LEADER
I20260808 16:52:22.045198 8288 external_mini_cluster.cc:1428] Setting key baa12df11e72563303362cedba1502b6
I20260808 16:52:22.045203 16325 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4141c4d302754d4b9c22b08ef5c4771f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4141c4d302754d4b9c22b08ef5c4771f. Latest consensus state: current_term: 1 leader_uuid: "4141c4d302754d4b9c22b08ef5c4771f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4141c4d302754d4b9c22b08ef5c4771f" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 37009 } } }
I20260808 16:52:22.045255 16325 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4141c4d302754d4b9c22b08ef5c4771f [sys.catalog]: This master's current role is: LEADER
I20260808 16:52:22.045593 16335 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 16:52:22.046074 16335 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 16:52:22.048086 16335 catalog_manager.cc:1347] Generated new cluster ID: cb68491ccc654bafa1b9fa820370281f
I20260808 16:52:22.048133 16335 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 16:52:22.050745 16347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:22.045697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56236 (local address 127.8.24.62:37009)
0808 16:52:22.046083 (+ 386us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:22.046091 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:22.046121 (+ 30us) server_negotiation.cc:407] Connection header received
0808 16:52:22.046227 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:22.046234 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:22.046297 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:22.046452 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:22.046659 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:22.047697 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:22.048788 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:22.049374 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:22.049473 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:22.050076 (+ 603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:22.050080 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:22.050080 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 16:52:22.050285 (+ 205us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:22.050321 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 16:52:22.050334 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:22.050412 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:22.050487 (+ 75us) server_negotiation.cc:299] Negotiation successful
0808 16:52:22.050555 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":62,"threads_started":1}
I20260808 16:52:22.051541 8288 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 16:52:22.061573 16335 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 16:52:22.062002 16335 catalog_manager.cc:1504] Loading token signing keys...
I20260808 16:52:22.066450 16335 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4141c4d302754d4b9c22b08ef5c4771f: Generated new TSK 0
I20260808 16:52:22.066579 16335 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260808 16:52:22.095757 16368 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260808 16:52:22.095865 16368 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260808 16:52:22.095969 16347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:22.073869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56250 (local address 127.8.24.62:37009)
0808 16:52:22.073909 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:22.073911 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:22.074031 (+ 120us) server_negotiation.cc:407] Connection header received
0808 16:52:22.074108 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:22.074109 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:22.074131 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:22.074185 (+ 54us) server_negotiation.cc:226] Negotiated authn=JWT
0808 16:52:22.074314 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:22.074804 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:22.075514 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:22.075707 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:22.075758 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:22.075939 (+ 181us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 16:52:22.095784 (+ 19845us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37839/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0808 16:52:22.095912 (+ 128us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:56250: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37839/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":23}
W20260808 16:52:22.096022 16347 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:56250: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37839/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260808 16:52:22.096496 16362 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:52:22.073805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:37009 (local address 127.0.0.1:56250)
0808 16:52:22.073972 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:22.073981 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:22.074040 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:22.074192 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:22.074193 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:22.074195 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0808 16:52:22.074259 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:22.074261 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:22.074872 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:22.074873 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:22.075445 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:22.075446 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:22.075762 (+ 316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:22.075765 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:22.075829 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:22.075847 (+ 18us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 16:52:22.096003 (+ 20156us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37839/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0808 16:52:22.096198 (+ 195us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.8.24.62:37009: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37839/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":76,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.8.24.62:37009: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37839/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260808 16:52:22.096865 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 16211
2026-08-08T16:52:22Z chronyd exiting
I20260808 16:52:22.144582 8288 test_util.cc:175] -----------------------------------------------
I20260808 16:52:22.144644 8288 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786207919384338-8288-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (253 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260808 16:52:22.145318 8288 mini_oidc.cc:152] Starting JWKS server
I20260808 16:52:22.145918 8288 webserver.cc:460] Webserver started at https://127.0.0.1:38841/ using document root <none> and password file <none>
I20260808 16:52:22.145957 8288 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 16:52:22.146060 8288 webserver.cc:460] Webserver started at http://127.0.0.1:40927/ using document root <none> and password file <none>
2026-08-08T16:52:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T16:52:22Z Disabled control of system clock
I20260808 16:52:22.153537 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786207919384338-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786207919384338-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786207919384338-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:33491
--webserver_interface=127.8.24.62
--webserver_port=0
--builtin_ntp_servers=127.8.24.20:35769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:33491
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38841/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786207919384338-8288-0/testchainca.cert with env {}
W20260808 16:52:22.236701 16375 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:52:22.236946 16375 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:52:22.236979 16375 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:52:22.238633 16375 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 16:52:22.238694 16375 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:52:22.238713 16375 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:52:22.238731 16375 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 16:52:22.238755 16375 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 16:52:22.238778 16375 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 16:52:22.238801 16375 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:38841/jwks.json
I20260808 16:52:22.240989 16375 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:35769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786207919384338-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786207919384338-8288-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:33491
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:33491
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38841/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.24.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786207919384338-8288-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16375
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786207919384338-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:52:22.241254 16375 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:52:22.241496 16375 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:52:22.247769 16387 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:22.247766 16385 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:22.248083 16384 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:22.248575 16375 server_base.cc:1034] running on GCE node
I20260808 16:52:22.248876 16375 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:52:22.249212 16375 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:52:22.250371 16375 hybrid_clock.cc:648] HybridClock initialized: now 1786207942250359 us; error 28 us; skew 500 ppm
I20260808 16:52:22.251667 16375 webserver.cc:460] Webserver started at http://127.8.24.62:35311/ using document root <none> and password file <none>
I20260808 16:52:22.251900 16375 fs_manager.cc:359] Metadata directory not provided
I20260808 16:52:22.251950 16375 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:52:22.252077 16375 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:52:22.253043 16375 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786207919384338-8288-0/minicluster-data/master-0/data/instance:
uuid: "384017019ba1458f84d3ecc1cb05eada"
format_stamp: "Formatted at 2026-08-08 16:52:22 on dist-test-slave-6zbq"
server_key: "3f5f6872c170c76b07679fa12185dea0"
server_key_iv: "739fa398bae540aa7651b6354b6ac257"
server_key_version: "encryptionkey@0"
I20260808 16:52:22.253381 16375 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786207919384338-8288-0/minicluster-data/master-0/wal/instance:
uuid: "384017019ba1458f84d3ecc1cb05eada"
format_stamp: "Formatted at 2026-08-08 16:52:22 on dist-test-slave-6zbq"
server_key: "3f5f6872c170c76b07679fa12185dea0"
server_key_iv: "739fa398bae540aa7651b6354b6ac257"
server_key_version: "encryptionkey@0"
I20260808 16:52:22.254736 16375 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 16:52:22.255587 16393 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:22.255745 16375 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:22.255844 16375 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786207919384338-8288-0/minicluster-data/master-0/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786207919384338-8288-0/minicluster-data/master-0/wal
uuid: "384017019ba1458f84d3ecc1cb05eada"
format_stamp: "Formatted at 2026-08-08 16:52:22 on dist-test-slave-6zbq"
server_key: "3f5f6872c170c76b07679fa12185dea0"
server_key_iv: "739fa398bae540aa7651b6354b6ac257"
server_key_version: "encryptionkey@0"
I20260808 16:52:22.255911 16375 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786207919384338-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786207919384338-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786207919384338-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:52:22.281452 16375 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:52:22.281702 16375 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:52:22.281829 16375 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:52:22.285387 16375 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.62:33491
I20260808 16:52:22.285415 16480 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.62:33491 every 8 connection(s)
I20260808 16:52:22.285701 16375 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
I20260808 16:52:22.286581 16481 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:52:22.287297 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 16375
I20260808 16:52:22.287395 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786207919384338-8288-0/minicluster-data/master-0/wal/instance
I20260808 16:52:22.287565 8288 external_mini_cluster.cc:1428] Setting key 15754258eb5aed412d4db58b0baff48a
I20260808 16:52:22.289373 16481 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 384017019ba1458f84d3ecc1cb05eada: Bootstrap starting.
I20260808 16:52:22.289937 16481 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 384017019ba1458f84d3ecc1cb05eada: Neither blocks nor log segments found. Creating new log.
I20260808 16:52:22.290228 16481 log.cc:824] T 00000000000000000000000000000000 P 384017019ba1458f84d3ecc1cb05eada: Log is configured to *not* fsync() on all Append() calls
I20260808 16:52:22.290956 16481 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 384017019ba1458f84d3ecc1cb05eada: No bootstrap required, opened a new log
I20260808 16:52:22.292595 16485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:22.288030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59440 (local address 127.8.24.62:33491)
0808 16:52:22.288210 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:22.288218 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:22.288241 (+ 23us) server_negotiation.cc:407] Connection header received
0808 16:52:22.288650 (+ 409us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:22.288656 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:22.288711 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:22.288820 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:22.289041 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:22.289918 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:22.290893 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:22.291310 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:22.291396 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:22.292025 (+ 629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:22.292029 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:22.292029 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 16:52:22.292174 (+ 145us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:22.292205 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 16:52:22.292215 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:22.292262 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:22.292347 (+ 85us) server_negotiation.cc:299] Negotiation successful
0808 16:52:22.292407 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":63,"threads_started":1}
I20260808 16:52:22.293002 16481 raft_consensus.cc:348] T 00000000000000000000000000000000 P 384017019ba1458f84d3ecc1cb05eada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "384017019ba1458f84d3ecc1cb05eada" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 33491 } }
I20260808 16:52:22.293157 16481 raft_consensus.cc:374] T 00000000000000000000000000000000 P 384017019ba1458f84d3ecc1cb05eada [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:52:22.293196 16481 raft_consensus.cc:729] T 00000000000000000000000000000000 P 384017019ba1458f84d3ecc1cb05eada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 384017019ba1458f84d3ecc1cb05eada, State: Initialized, Role: FOLLOWER
I20260808 16:52:22.293299 16481 consensus_queue.cc:260] T 00000000000000000000000000000000 P 384017019ba1458f84d3ecc1cb05eada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "384017019ba1458f84d3ecc1cb05eada" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 33491 } }
I20260808 16:52:22.293372 16481 raft_consensus.cc:388] T 00000000000000000000000000000000 P 384017019ba1458f84d3ecc1cb05eada [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 16:52:22.293395 16481 raft_consensus.cc:482] T 00000000000000000000000000000000 P 384017019ba1458f84d3ecc1cb05eada [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 16:52:22.293434 16481 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 384017019ba1458f84d3ecc1cb05eada [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:52:22.293516 8288 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 16:52:22.294054 16481 raft_consensus.cc:504] T 00000000000000000000000000000000 P 384017019ba1458f84d3ecc1cb05eada [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "384017019ba1458f84d3ecc1cb05eada" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 33491 } }
I20260808 16:52:22.294173 16481 leader_election.cc:302] T 00000000000000000000000000000000 P 384017019ba1458f84d3ecc1cb05eada [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 384017019ba1458f84d3ecc1cb05eada; no voters:
I20260808 16:52:22.294361 16481 leader_election.cc:288] T 00000000000000000000000000000000 P 384017019ba1458f84d3ecc1cb05eada [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 16:52:22.294414 16488 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 384017019ba1458f84d3ecc1cb05eada [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:52:22.294559 16488 raft_consensus.cc:686] T 00000000000000000000000000000000 P 384017019ba1458f84d3ecc1cb05eada [term 1 LEADER]: Becoming Leader. State: Replica: 384017019ba1458f84d3ecc1cb05eada, State: Running, Role: LEADER
I20260808 16:52:22.294600 16481 sys_catalog.cc:564] T 00000000000000000000000000000000 P 384017019ba1458f84d3ecc1cb05eada [sys.catalog]: configured and running, proceeding with master startup.
I20260808 16:52:22.294643 16488 consensus_queue.cc:237] T 00000000000000000000000000000000 P 384017019ba1458f84d3ecc1cb05eada [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "384017019ba1458f84d3ecc1cb05eada" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 33491 } }
I20260808 16:52:22.294960 16488 sys_catalog.cc:455] T 00000000000000000000000000000000 P 384017019ba1458f84d3ecc1cb05eada [sys.catalog]: SysCatalogTable state changed. Reason: New leader 384017019ba1458f84d3ecc1cb05eada. Latest consensus state: current_term: 1 leader_uuid: "384017019ba1458f84d3ecc1cb05eada" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "384017019ba1458f84d3ecc1cb05eada" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 33491 } } }
I20260808 16:52:22.295039 16488 sys_catalog.cc:458] T 00000000000000000000000000000000 P 384017019ba1458f84d3ecc1cb05eada [sys.catalog]: This master's current role is: LEADER
I20260808 16:52:22.294996 16491 sys_catalog.cc:455] T 00000000000000000000000000000000 P 384017019ba1458f84d3ecc1cb05eada [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "384017019ba1458f84d3ecc1cb05eada" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "384017019ba1458f84d3ecc1cb05eada" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 33491 } } }
I20260808 16:52:22.295207 16491 sys_catalog.cc:458] T 00000000000000000000000000000000 P 384017019ba1458f84d3ecc1cb05eada [sys.catalog]: This master's current role is: LEADER
I20260808 16:52:22.295536 16506 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 16:52:22.296022 16506 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 16:52:22.297394 16506 catalog_manager.cc:1347] Generated new cluster ID: e20b9dbf39b24a7f874ac914daa95de1
I20260808 16:52:22.297441 16506 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 16:52:22.314314 16506 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 16:52:22.314786 16506 catalog_manager.cc:1504] Loading token signing keys...
I20260808 16:52:22.319012 16506 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 384017019ba1458f84d3ecc1cb05eada: Generated new TSK 0
I20260808 16:52:22.319162 16506 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260808 16:52:22.340598 16534 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260808 16:52:22.340660 16534 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260808 16:52:22.340907 16485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:22.327233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59448 (local address 127.8.24.62:33491)
0808 16:52:22.327279 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:22.327281 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:22.327334 (+ 53us) server_negotiation.cc:407] Connection header received
0808 16:52:22.327413 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:22.327413 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:22.327422 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:22.327474 (+ 52us) server_negotiation.cc:226] Negotiated authn=JWT
0808 16:52:22.327588 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:22.328055 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:22.329107 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:22.329293 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:22.329333 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:22.329484 (+ 151us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 16:52:22.340758 (+ 11274us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38841/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0808 16:52:22.340864 (+ 106us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:59448: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38841/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":20}
W20260808 16:52:22.340947 16485 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:59448: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38841/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260808 16:52:22.341053 16532 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:52:22.327153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:33491 (local address 127.0.0.1:59448)
0808 16:52:22.327280 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:22.327290 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:22.327350 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:22.327483 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:22.327483 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:22.327485 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0808 16:52:22.327536 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:22.327538 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:22.328114 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:22.328115 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:22.329042 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:22.329045 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:22.329346 (+ 301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:22.329350 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:22.329409 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:22.329414 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 16:52:22.340848 (+ 11434us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38841/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0808 16:52:22.340957 (+ 109us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.8.24.62:33491: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38841/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":75,"threads_started":1}
I20260808 16:52:22.341300 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 16375
2026-08-08T16:52:22Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (238 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260808 16:52:22.383628 8288 mini_oidc.cc:152] Starting JWKS server
I20260808 16:52:22.384223 8288 webserver.cc:460] Webserver started at https://127.0.0.1:43659/ using document root <none> and password file <none>
I20260808 16:52:22.384263 8288 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 16:52:22.384410 8288 webserver.cc:460] Webserver started at http://127.0.0.1:39941/ using document root <none> and password file <none>
2026-08-08T16:52:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T16:52:22Z Disabled control of system clock
I20260808 16:52:22.391764 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786207919384338-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786207919384338-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786207919384338-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:40975
--webserver_interface=127.8.24.62
--webserver_port=0
--builtin_ntp_servers=127.8.24.20:35455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:40975
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43659/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260808 16:52:22.468237 16542 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:52:22.468405 16542 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:52:22.468423 16542 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:52:22.469515 16542 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 16:52:22.469553 16542 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:52:22.469564 16542 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:52:22.469576 16542 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 16:52:22.469600 16542 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 16:52:22.469623 16542 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 16:52:22.469645 16542 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:43659/jwks.json
I20260808 16:52:22.470968 16542 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:35455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786207919384338-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786207919384338-8288-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:40975
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:40975
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43659/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.24.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16542
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786207919384338-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:52:22.471179 16542 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:52:22.471421 16542 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:52:22.475642 16555 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:22.475667 16552 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:22.475795 16542 server_base.cc:1034] running on GCE node
W20260808 16:52:22.475927 16549 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:22.476194 16542 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:52:22.476464 16542 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:52:22.477597 16542 hybrid_clock.cc:648] HybridClock initialized: now 1786207942477583 us; error 31 us; skew 500 ppm
I20260808 16:52:22.478688 16542 webserver.cc:460] Webserver started at http://127.8.24.62:44259/ using document root <none> and password file <none>
I20260808 16:52:22.478884 16542 fs_manager.cc:359] Metadata directory not provided
I20260808 16:52:22.478930 16542 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:52:22.479045 16542 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:52:22.479974 16542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786207919384338-8288-0/minicluster-data/master-0/data/instance:
uuid: "0d09ed45100a4bb79bb6371fd913abd9"
format_stamp: "Formatted at 2026-08-08 16:52:22 on dist-test-slave-6zbq"
server_key: "d66d8cb631ec70b1dc35d7d1edf90feb"
server_key_iv: "888cbd49b0d342d8da4dda1da79bc8f8"
server_key_version: "encryptionkey@0"
I20260808 16:52:22.480288 16542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786207919384338-8288-0/minicluster-data/master-0/wal/instance:
uuid: "0d09ed45100a4bb79bb6371fd913abd9"
format_stamp: "Formatted at 2026-08-08 16:52:22 on dist-test-slave-6zbq"
server_key: "d66d8cb631ec70b1dc35d7d1edf90feb"
server_key_iv: "888cbd49b0d342d8da4dda1da79bc8f8"
server_key_version: "encryptionkey@0"
I20260808 16:52:22.481423 16542 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 16:52:22.482014 16563 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:22.482153 16542 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 16:52:22.482232 16542 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786207919384338-8288-0/minicluster-data/master-0/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786207919384338-8288-0/minicluster-data/master-0/wal
uuid: "0d09ed45100a4bb79bb6371fd913abd9"
format_stamp: "Formatted at 2026-08-08 16:52:22 on dist-test-slave-6zbq"
server_key: "d66d8cb631ec70b1dc35d7d1edf90feb"
server_key_iv: "888cbd49b0d342d8da4dda1da79bc8f8"
server_key_version: "encryptionkey@0"
I20260808 16:52:22.482297 16542 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786207919384338-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786207919384338-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786207919384338-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:52:22.492187 16542 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:52:22.492429 16542 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:52:22.492543 16542 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:52:22.496031 16542 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.62:40975
I20260808 16:52:22.496062 16652 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.62:40975 every 8 connection(s)
I20260808 16:52:22.496330 16542 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
I20260808 16:52:22.496884 16653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:52:22.499181 16653 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d09ed45100a4bb79bb6371fd913abd9: Bootstrap starting.
I20260808 16:52:22.499650 16653 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0d09ed45100a4bb79bb6371fd913abd9: Neither blocks nor log segments found. Creating new log.
I20260808 16:52:22.499881 16653 log.cc:824] T 00000000000000000000000000000000 P 0d09ed45100a4bb79bb6371fd913abd9: Log is configured to *not* fsync() on all Append() calls
I20260808 16:52:22.500422 16653 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d09ed45100a4bb79bb6371fd913abd9: No bootstrap required, opened a new log
I20260808 16:52:22.501873 16653 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0d09ed45100a4bb79bb6371fd913abd9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d09ed45100a4bb79bb6371fd913abd9" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 40975 } }
I20260808 16:52:22.501991 16653 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0d09ed45100a4bb79bb6371fd913abd9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:52:22.502022 16653 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0d09ed45100a4bb79bb6371fd913abd9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d09ed45100a4bb79bb6371fd913abd9, State: Initialized, Role: FOLLOWER
I20260808 16:52:22.502110 16653 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0d09ed45100a4bb79bb6371fd913abd9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d09ed45100a4bb79bb6371fd913abd9" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 40975 } }
I20260808 16:52:22.502158 16653 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0d09ed45100a4bb79bb6371fd913abd9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 16:52:22.502179 16653 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0d09ed45100a4bb79bb6371fd913abd9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 16:52:22.502207 16653 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0d09ed45100a4bb79bb6371fd913abd9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:52:22.502682 16653 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0d09ed45100a4bb79bb6371fd913abd9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d09ed45100a4bb79bb6371fd913abd9" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 40975 } }
I20260808 16:52:22.502764 16653 leader_election.cc:302] T 00000000000000000000000000000000 P 0d09ed45100a4bb79bb6371fd913abd9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0d09ed45100a4bb79bb6371fd913abd9; no voters:
I20260808 16:52:22.502876 16653 leader_election.cc:288] T 00000000000000000000000000000000 P 0d09ed45100a4bb79bb6371fd913abd9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 16:52:22.502921 16661 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0d09ed45100a4bb79bb6371fd913abd9 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:52:22.503077 16653 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0d09ed45100a4bb79bb6371fd913abd9 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 16:52:22.503098 16661 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0d09ed45100a4bb79bb6371fd913abd9 [term 1 LEADER]: Becoming Leader. State: Replica: 0d09ed45100a4bb79bb6371fd913abd9, State: Running, Role: LEADER
I20260808 16:52:22.503193 16661 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0d09ed45100a4bb79bb6371fd913abd9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d09ed45100a4bb79bb6371fd913abd9" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 40975 } }
I20260808 16:52:22.503629 16664 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d09ed45100a4bb79bb6371fd913abd9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0d09ed45100a4bb79bb6371fd913abd9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d09ed45100a4bb79bb6371fd913abd9" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 40975 } } }
I20260808 16:52:22.503718 16664 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d09ed45100a4bb79bb6371fd913abd9 [sys.catalog]: This master's current role is: LEADER
I20260808 16:52:22.503731 16665 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d09ed45100a4bb79bb6371fd913abd9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0d09ed45100a4bb79bb6371fd913abd9. Latest consensus state: current_term: 1 leader_uuid: "0d09ed45100a4bb79bb6371fd913abd9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d09ed45100a4bb79bb6371fd913abd9" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 40975 } } }
I20260808 16:52:22.503787 16665 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d09ed45100a4bb79bb6371fd913abd9 [sys.catalog]: This master's current role is: LEADER
I20260808 16:52:22.503916 16676 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 16:52:22.504367 16676 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 16:52:22.505281 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 16542
I20260808 16:52:22.505373 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786207919384338-8288-0/minicluster-data/master-0/wal/instance
I20260808 16:52:22.505558 8288 external_mini_cluster.cc:1428] Setting key fc47a69c1bc65a9bf61ffdfbc7d325c1
I20260808 16:52:22.505852 16676 catalog_manager.cc:1347] Generated new cluster ID: 3de437bb9fe24cf9b837067553ba5a6d
I20260808 16:52:22.505897 16676 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 16:52:22.515487 16697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:22.505919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58152 (local address 127.8.24.62:40975)
0808 16:52:22.506101 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:22.506109 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:22.510599 (+ 4490us) server_negotiation.cc:407] Connection header received
0808 16:52:22.510733 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:22.510740 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:22.510808 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:22.510946 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:22.511090 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:22.512442 (+ 1352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:22.513427 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:22.514018 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:22.514133 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:22.514752 (+ 619us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:22.514756 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:22.514756 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 16:52:22.514939 (+ 183us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:22.514977 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 16:52:22.514991 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:22.515057 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:22.515111 (+ 54us) server_negotiation.cc:299] Negotiation successful
0808 16:52:22.515280 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":77,"threads_started":1}
I20260808 16:52:22.516328 8288 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 16:52:22.557374 16676 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 16:52:22.558034 16676 catalog_manager.cc:1504] Loading token signing keys...
I20260808 16:52:22.565656 16676 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0d09ed45100a4bb79bb6371fd913abd9: Generated new TSK 0
I20260808 16:52:22.565807 16676 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 16:52:22.587414 16697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:22.582958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58166 (local address 127.8.24.62:40975)
0808 16:52:22.583001 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:22.583003 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:22.583094 (+ 91us) server_negotiation.cc:407] Connection header received
0808 16:52:22.583133 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:22.583134 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:22.583146 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:22.583217 (+ 71us) server_negotiation.cc:226] Negotiated authn=JWT
0808 16:52:22.583340 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:22.584077 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:22.584870 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:22.585179 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:22.585247 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:22.585414 (+ 167us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 16:52:22.587212 (+ 1798us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43659/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0808 16:52:22.587349 (+ 137us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:58166: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43659/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":25}
W20260808 16:52:22.587469 16697 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:58166: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43659/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260808 16:52:22.589257 16709 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:52:22.582895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:40975 (local address 127.0.0.1:58166)
0808 16:52:22.583024 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:22.583034 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:22.583089 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:22.583224 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:22.583225 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:22.583227 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0808 16:52:22.583284 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:22.583286 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:22.584165 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:22.584166 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:22.584790 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:22.584795 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:22.585257 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:22.585261 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:22.585327 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:22.585331 (+ 4us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 16:52:22.587324 (+ 1993us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43659/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0808 16:52:22.589162 (+ 1838us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.8.24.62:40975: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43659/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":107,"mutex_wait_us":1684,"thread_start_us":74,"threads_started":1}
I20260808 16:52:22.589507 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 16542
E20260808 16:52:22.608408 16710 webserver.cc:533] Webserver: sslize failed:
E20260808 16:52:22.608480 16710 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-08T16:52:22Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (253 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-08T16:52:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T16:52:22Z Disabled control of system clock
I20260808 16:52:22.645270 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786207919384338-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786207919384338-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786207919384338-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:41813
--webserver_interface=127.8.24.62
--webserver_port=0
--builtin_ntp_servers=127.8.24.20:39547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:41813
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786207919384338-8288-0/testchainca.cert with env {}
W20260808 16:52:22.722308 16721 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:52:22.722504 16721 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:52:22.722524 16721 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:52:22.723644 16721 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 16:52:22.723682 16721 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:52:22.723694 16721 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:52:22.723716 16721 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 16:52:22.723739 16721 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 16:52:22.723762 16721 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 16:52:22.723776 16721 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260808 16:52:22.725209 16721 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:39547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786207919384338-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786207919384338-8288-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:41813
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:41813
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.24.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786207919384338-8288-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16721
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786207919384338-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:52:22.725432 16721 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:52:22.725616 16721 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:52:22.729964 16735 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:22.730046 16721 server_base.cc:1034] running on GCE node
W20260808 16:52:22.729996 16733 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:22.729967 16738 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:22.730474 16721 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:52:22.730722 16721 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:52:22.731853 16721 hybrid_clock.cc:648] HybridClock initialized: now 1786207942731840 us; error 30 us; skew 500 ppm
I20260808 16:52:22.732919 16721 webserver.cc:460] Webserver started at http://127.8.24.62:37137/ using document root <none> and password file <none>
I20260808 16:52:22.733129 16721 fs_manager.cc:359] Metadata directory not provided
I20260808 16:52:22.733172 16721 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:52:22.733286 16721 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:52:22.734225 16721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786207919384338-8288-0/minicluster-data/master-0/data/instance:
uuid: "eb9427807ec44fb4aec60b5e62fc5700"
format_stamp: "Formatted at 2026-08-08 16:52:22 on dist-test-slave-6zbq"
server_key: "3e7a477fcbe23b1be4e1206041ed7c45"
server_key_iv: "62cdd6514ff591dae16c4df54d9e0ec2"
server_key_version: "encryptionkey@0"
I20260808 16:52:22.734565 16721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786207919384338-8288-0/minicluster-data/master-0/wal/instance:
uuid: "eb9427807ec44fb4aec60b5e62fc5700"
format_stamp: "Formatted at 2026-08-08 16:52:22 on dist-test-slave-6zbq"
server_key: "3e7a477fcbe23b1be4e1206041ed7c45"
server_key_iv: "62cdd6514ff591dae16c4df54d9e0ec2"
server_key_version: "encryptionkey@0"
I20260808 16:52:22.735694 16721 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 16:52:22.736442 16745 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:22.736586 16721 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 16:52:22.736682 16721 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786207919384338-8288-0/minicluster-data/master-0/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786207919384338-8288-0/minicluster-data/master-0/wal
uuid: "eb9427807ec44fb4aec60b5e62fc5700"
format_stamp: "Formatted at 2026-08-08 16:52:22 on dist-test-slave-6zbq"
server_key: "3e7a477fcbe23b1be4e1206041ed7c45"
server_key_iv: "62cdd6514ff591dae16c4df54d9e0ec2"
server_key_version: "encryptionkey@0"
I20260808 16:52:22.736750 16721 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786207919384338-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786207919384338-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786207919384338-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:52:22.751961 16721 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:52:22.752205 16721 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:52:22.752326 16721 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:52:22.755983 16721 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.62:41813
I20260808 16:52:22.756012 16829 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.62:41813 every 8 connection(s)
I20260808 16:52:22.756314 16721 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
I20260808 16:52:22.757174 16831 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:52:22.759073 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 16721
I20260808 16:52:22.759186 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786207919384338-8288-0/minicluster-data/master-0/wal/instance
I20260808 16:52:22.759389 8288 external_mini_cluster.cc:1428] Setting key 14506d55e1c81131cecb0a4a6bc7566f
I20260808 16:52:22.759501 16831 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb9427807ec44fb4aec60b5e62fc5700: Bootstrap starting.
I20260808 16:52:22.760133 16831 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eb9427807ec44fb4aec60b5e62fc5700: Neither blocks nor log segments found. Creating new log.
I20260808 16:52:22.760399 16831 log.cc:824] T 00000000000000000000000000000000 P eb9427807ec44fb4aec60b5e62fc5700: Log is configured to *not* fsync() on all Append() calls
I20260808 16:52:22.760998 16831 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb9427807ec44fb4aec60b5e62fc5700: No bootstrap required, opened a new log
I20260808 16:52:22.762735 16831 raft_consensus.cc:348] T 00000000000000000000000000000000 P eb9427807ec44fb4aec60b5e62fc5700 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb9427807ec44fb4aec60b5e62fc5700" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 41813 } }
I20260808 16:52:22.762871 16831 raft_consensus.cc:374] T 00000000000000000000000000000000 P eb9427807ec44fb4aec60b5e62fc5700 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:52:22.762893 16831 raft_consensus.cc:729] T 00000000000000000000000000000000 P eb9427807ec44fb4aec60b5e62fc5700 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb9427807ec44fb4aec60b5e62fc5700, State: Initialized, Role: FOLLOWER
I20260808 16:52:22.762996 16831 consensus_queue.cc:260] T 00000000000000000000000000000000 P eb9427807ec44fb4aec60b5e62fc5700 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb9427807ec44fb4aec60b5e62fc5700" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 41813 } }
I20260808 16:52:22.763054 16831 raft_consensus.cc:388] T 00000000000000000000000000000000 P eb9427807ec44fb4aec60b5e62fc5700 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 16:52:22.763077 16831 raft_consensus.cc:482] T 00000000000000000000000000000000 P eb9427807ec44fb4aec60b5e62fc5700 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 16:52:22.763121 16831 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eb9427807ec44fb4aec60b5e62fc5700 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:52:22.763644 16831 raft_consensus.cc:504] T 00000000000000000000000000000000 P eb9427807ec44fb4aec60b5e62fc5700 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb9427807ec44fb4aec60b5e62fc5700" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 41813 } }
I20260808 16:52:22.763748 16831 leader_election.cc:302] T 00000000000000000000000000000000 P eb9427807ec44fb4aec60b5e62fc5700 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eb9427807ec44fb4aec60b5e62fc5700; no voters:
I20260808 16:52:22.763909 16831 leader_election.cc:288] T 00000000000000000000000000000000 P eb9427807ec44fb4aec60b5e62fc5700 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 16:52:22.763936 16838 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eb9427807ec44fb4aec60b5e62fc5700 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:52:22.764066 16838 raft_consensus.cc:686] T 00000000000000000000000000000000 P eb9427807ec44fb4aec60b5e62fc5700 [term 1 LEADER]: Becoming Leader. State: Replica: eb9427807ec44fb4aec60b5e62fc5700, State: Running, Role: LEADER
I20260808 16:52:22.764155 16831 sys_catalog.cc:564] T 00000000000000000000000000000000 P eb9427807ec44fb4aec60b5e62fc5700 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 16:52:22.764164 16838 consensus_queue.cc:237] T 00000000000000000000000000000000 P eb9427807ec44fb4aec60b5e62fc5700 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb9427807ec44fb4aec60b5e62fc5700" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 41813 } }
I20260808 16:52:22.764496 16838 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb9427807ec44fb4aec60b5e62fc5700 [sys.catalog]: SysCatalogTable state changed. Reason: New leader eb9427807ec44fb4aec60b5e62fc5700. Latest consensus state: current_term: 1 leader_uuid: "eb9427807ec44fb4aec60b5e62fc5700" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb9427807ec44fb4aec60b5e62fc5700" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 41813 } } }
I20260808 16:52:22.764613 16840 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb9427807ec44fb4aec60b5e62fc5700 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eb9427807ec44fb4aec60b5e62fc5700" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb9427807ec44fb4aec60b5e62fc5700" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 41813 } } }
I20260808 16:52:22.764681 16840 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb9427807ec44fb4aec60b5e62fc5700 [sys.catalog]: This master's current role is: LEADER
I20260808 16:52:22.764771 16836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:22.759833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58634 (local address 127.8.24.62:41813)
0808 16:52:22.760013 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:22.760020 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:22.760042 (+ 22us) server_negotiation.cc:407] Connection header received
0808 16:52:22.760477 (+ 435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:22.760486 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:22.760549 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:22.760676 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:22.760875 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:22.761774 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:22.762587 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:22.763057 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:22.763175 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:22.763785 (+ 610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:22.763790 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:22.763790 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 16:52:22.763980 (+ 190us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:22.764024 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 16:52:22.764039 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:22.764398 (+ 359us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:22.764431 (+ 33us) server_negotiation.cc:299] Negotiation successful
0808 16:52:22.764507 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":64,"threads_started":1}
I20260808 16:52:22.764577 16838 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb9427807ec44fb4aec60b5e62fc5700 [sys.catalog]: This master's current role is: LEADER
I20260808 16:52:22.764950 16850 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 16:52:22.765527 16850 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 16:52:22.765620 8288 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 16:52:22.766938 16850 catalog_manager.cc:1347] Generated new cluster ID: ac58f1a785434e3abe81b9410ade00fd
I20260808 16:52:22.766984 16850 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 16:52:22.781847 16850 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 16:52:22.782327 16850 catalog_manager.cc:1504] Loading token signing keys...
I20260808 16:52:22.785527 16850 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eb9427807ec44fb4aec60b5e62fc5700: Generated new TSK 0
I20260808 16:52:22.785683 16850 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 16:52:22.953333 16836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:22.796079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58638 (local address 127.8.24.62:41813)
0808 16:52:22.796139 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:22.796141 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:22.796246 (+ 105us) server_negotiation.cc:407] Connection header received
0808 16:52:22.796322 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:22.796322 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:22.796332 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:22.796388 (+ 56us) server_negotiation.cc:226] Negotiated authn=JWT
0808 16:52:22.796523 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:22.797030 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:22.798144 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:22.798373 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:22.798435 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:22.798607 (+ 172us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 16:52:22.953117 (+154510us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0808 16:52:22.953273 (+ 156us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:58638: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":36}
W20260808 16:52:22.953403 16836 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:58638: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260808 16:52:22.953469 16871 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:52:22.796020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:41813 (local address 127.0.0.1:58638)
0808 16:52:22.796179 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:22.796190 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:22.796257 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:22.796408 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:22.796408 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:22.796410 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0808 16:52:22.796470 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:22.796472 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:22.797104 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:22.797105 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:22.798048 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:22.798052 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:22.798445 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:22.798449 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:22.798520 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:22.798526 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 16:52:22.953273 (+154747us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0808 16:52:22.953383 (+ 110us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.8.24.62:41813: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":77,"threads_started":1}
I20260808 16:52:22.953850 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 16721
2026-08-08T16:52:22Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (339 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (86 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (84 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786207919384338-8288-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 16:52:23 dist-test-slave-6zbq krb5kdc[16893](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 16:52:23 dist-test-slave-6zbq krb5kdc[16893](info): set up 2 sockets
Aug 08 16:52:23 dist-test-slave-6zbq krb5kdc[16893](info): commencing operation
krb5kdc: starting...
W20260808 16:52:24.247068 8288 mini_kdc.cc:121] Time spent starting KDC: real 1.099s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786207919384338-8288-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786207919384338-8288-0/krb5kdc/test-user.keytab.
Aug 08 16:52:24 dist-test-slave-6zbq krb5kdc[16893](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207944, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T16:52:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T16:52:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.24.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
I20260808 16:52:24.352494 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786207919384338-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786207919384338-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786207919384338-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:38071
--webserver_interface=127.8.24.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:38273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:38071
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:52:24.430442 16919 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:52:24.430624 16919 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:52:24.430644 16919 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:52:24.431734 16919 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 16:52:24.431767 16919 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:52:24.431780 16919 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:52:24.431792 16919 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 16:52:24.431802 16919 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 16:52:24.433166 16919 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:38273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786207919384338-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786207919384338-8288-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:38071
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:38071
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.24.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16919
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786207919384338-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:52:24.433346 16919 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:52:24.433503 16919 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:52:24.437885 16927 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:24.437963 16919 server_base.cc:1034] running on GCE node
W20260808 16:52:24.437881 16926 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:24.437914 16930 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:24.438380 16919 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:52:24.438664 16919 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:52:24.439803 16919 hybrid_clock.cc:648] HybridClock initialized: now 1786207944439789 us; error 30 us; skew 500 ppm
Aug 08 16:52:24 dist-test-slave-6zbq krb5kdc[16893](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207944, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:52:24.442101 16919 init.cc:377] Logged in from keytab as kudu/127.8.24.62@KRBTEST.COM (short username kudu)
I20260808 16:52:24.442533 16919 webserver.cc:460] Webserver started at http://127.8.24.62:33899/ using document root <none> and password file <none>
I20260808 16:52:24.442740 16919 fs_manager.cc:359] Metadata directory not provided
I20260808 16:52:24.442785 16919 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:52:24.442899 16919 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:52:24.443817 16919 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786207919384338-8288-0/minicluster-data/master-0/data/instance:
uuid: "f8e8e2c945a144ed87cc2f4a20f6635e"
format_stamp: "Formatted at 2026-08-08 16:52:24 on dist-test-slave-6zbq"
server_key: "1713584d214aecf3f3fac64289401a22"
server_key_iv: "085179d95f0efd9eabad5ee7c55f8a43"
server_key_version: "encryptionkey@0"
I20260808 16:52:24.444125 16919 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786207919384338-8288-0/minicluster-data/master-0/wal/instance:
uuid: "f8e8e2c945a144ed87cc2f4a20f6635e"
format_stamp: "Formatted at 2026-08-08 16:52:24 on dist-test-slave-6zbq"
server_key: "1713584d214aecf3f3fac64289401a22"
server_key_iv: "085179d95f0efd9eabad5ee7c55f8a43"
server_key_version: "encryptionkey@0"
I20260808 16:52:24.445381 16919 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 16:52:24.446067 16943 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:24.446223 16919 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 16:52:24.446305 16919 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786207919384338-8288-0/minicluster-data/master-0/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786207919384338-8288-0/minicluster-data/master-0/wal
uuid: "f8e8e2c945a144ed87cc2f4a20f6635e"
format_stamp: "Formatted at 2026-08-08 16:52:24 on dist-test-slave-6zbq"
server_key: "1713584d214aecf3f3fac64289401a22"
server_key_iv: "085179d95f0efd9eabad5ee7c55f8a43"
server_key_version: "encryptionkey@0"
I20260808 16:52:24.446391 16919 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786207919384338-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786207919384338-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786207919384338-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:52:24.457262 16919 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:52:24.458971 16919 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:52:24.459091 16919 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:52:24.462581 16919 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.62:38071
I20260808 16:52:24.462620 17035 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.62:38071 every 8 connection(s)
I20260808 16:52:24.462908 16919 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
I20260808 16:52:24.463667 17036 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:52:24.465656 17036 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8e8e2c945a144ed87cc2f4a20f6635e: Bootstrap starting.
I20260808 16:52:24.466135 17036 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f8e8e2c945a144ed87cc2f4a20f6635e: Neither blocks nor log segments found. Creating new log.
I20260808 16:52:24.466372 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 16919
I20260808 16:52:24.466465 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786207919384338-8288-0/minicluster-data/master-0/wal/instance
I20260808 16:52:24.466449 17036 log.cc:824] T 00000000000000000000000000000000 P f8e8e2c945a144ed87cc2f4a20f6635e: Log is configured to *not* fsync() on all Append() calls
I20260808 16:52:24.466637 8288 external_mini_cluster.cc:1428] Setting key 3d3972670b60c6d9d9d0ec68a36a3008
I20260808 16:52:24.467152 17036 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8e8e2c945a144ed87cc2f4a20f6635e: No bootstrap required, opened a new log
I20260808 16:52:24.469154 17036 raft_consensus.cc:348] T 00000000000000000000000000000000 P f8e8e2c945a144ed87cc2f4a20f6635e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8e8e2c945a144ed87cc2f4a20f6635e" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 38071 } }
I20260808 16:52:24.469280 17036 raft_consensus.cc:374] T 00000000000000000000000000000000 P f8e8e2c945a144ed87cc2f4a20f6635e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:52:24.469303 17036 raft_consensus.cc:729] T 00000000000000000000000000000000 P f8e8e2c945a144ed87cc2f4a20f6635e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8e8e2c945a144ed87cc2f4a20f6635e, State: Initialized, Role: FOLLOWER
I20260808 16:52:24.469424 17036 consensus_queue.cc:260] T 00000000000000000000000000000000 P f8e8e2c945a144ed87cc2f4a20f6635e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8e8e2c945a144ed87cc2f4a20f6635e" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 38071 } }
I20260808 16:52:24.469493 17036 raft_consensus.cc:388] T 00000000000000000000000000000000 P f8e8e2c945a144ed87cc2f4a20f6635e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 16:52:24.469515 17036 raft_consensus.cc:482] T 00000000000000000000000000000000 P f8e8e2c945a144ed87cc2f4a20f6635e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 16:52:24.469558 17036 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f8e8e2c945a144ed87cc2f4a20f6635e [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:52:24.470049 17036 raft_consensus.cc:504] T 00000000000000000000000000000000 P f8e8e2c945a144ed87cc2f4a20f6635e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8e8e2c945a144ed87cc2f4a20f6635e" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 38071 } }
I20260808 16:52:24.470139 17036 leader_election.cc:302] T 00000000000000000000000000000000 P f8e8e2c945a144ed87cc2f4a20f6635e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f8e8e2c945a144ed87cc2f4a20f6635e; no voters:
I20260808 16:52:24.470293 17036 leader_election.cc:288] T 00000000000000000000000000000000 P f8e8e2c945a144ed87cc2f4a20f6635e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 16:52:24.470363 17044 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f8e8e2c945a144ed87cc2f4a20f6635e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:52:24.470515 17044 raft_consensus.cc:686] T 00000000000000000000000000000000 P f8e8e2c945a144ed87cc2f4a20f6635e [term 1 LEADER]: Becoming Leader. State: Replica: f8e8e2c945a144ed87cc2f4a20f6635e, State: Running, Role: LEADER
I20260808 16:52:24.470549 17036 sys_catalog.cc:564] T 00000000000000000000000000000000 P f8e8e2c945a144ed87cc2f4a20f6635e [sys.catalog]: configured and running, proceeding with master startup.
I20260808 16:52:24.470597 17044 consensus_queue.cc:237] T 00000000000000000000000000000000 P f8e8e2c945a144ed87cc2f4a20f6635e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8e8e2c945a144ed87cc2f4a20f6635e" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 38071 } }
I20260808 16:52:24.470968 17044 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8e8e2c945a144ed87cc2f4a20f6635e [sys.catalog]: SysCatalogTable state changed. Reason: New leader f8e8e2c945a144ed87cc2f4a20f6635e. Latest consensus state: current_term: 1 leader_uuid: "f8e8e2c945a144ed87cc2f4a20f6635e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8e8e2c945a144ed87cc2f4a20f6635e" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 38071 } } }
I20260808 16:52:24.471055 17045 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8e8e2c945a144ed87cc2f4a20f6635e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f8e8e2c945a144ed87cc2f4a20f6635e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8e8e2c945a144ed87cc2f4a20f6635e" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 38071 } } }
I20260808 16:52:24.471166 17045 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8e8e2c945a144ed87cc2f4a20f6635e [sys.catalog]: This master's current role is: LEADER
I20260808 16:52:24.471349 17052 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 16:52:24.471047 17044 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8e8e2c945a144ed87cc2f4a20f6635e [sys.catalog]: This master's current role is: LEADER
I20260808 16:52:24.471832 17052 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 08 16:52:24 dist-test-slave-6zbq krb5kdc[16893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207944, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260808 16:52:24.474438 17052 catalog_manager.cc:1347] Generated new cluster ID: 893232b44fd0448c946e65e64b0d567c
I20260808 16:52:24.474483 17052 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 16:52:24.477744 17042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:24.467077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45730 (local address 127.8.24.62:38071)
0808 16:52:24.467283 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:24.467290 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:24.467314 (+ 24us) server_negotiation.cc:407] Connection header received
0808 16:52:24.468517 (+ 1203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:24.468527 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:24.468580 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:24.468697 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:24.469281 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:24.470607 (+ 1326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:24.471660 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:24.472216 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:24.472609 (+ 393us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:24.474870 (+ 2261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:24.474874 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:24.474875 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:24.474886 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:24.476314 (+ 1428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:24.476629 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:24.476632 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:24.476633 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:24.476728 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:24.476900 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:24.476901 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:24.476901 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:24.477111 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:24.477245 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:24.477351 (+ 106us) server_negotiation.cc:299] Negotiation successful
0808 16:52:24.477476 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":86,"threads_started":1}
I20260808 16:52:24.480284 17052 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 16:52:24.480932 17052 catalog_manager.cc:1504] Loading token signing keys...
I20260808 16:52:24.484330 17052 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f8e8e2c945a144ed87cc2f4a20f6635e: Generated new TSK 0
I20260808 16:52:24.484478 17052 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.24.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.1.keytab.
Entry for principal kudu/127.8.24.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.1.keytab.
I20260808 16:52:24.512960 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786207919384338-8288-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786207919384338-8288-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786207919384338-8288-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786207919384338-8288-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.1:0
--local_ip_for_outbound_sockets=127.8.24.1
--webserver_interface=127.8.24.1
--webserver_port=0
--tserver_master_addrs=127.8.24.62:38071
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.1.keytab
--principal=kudu/127.8.24.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:38273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:52:24.617782 17073 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:52:24.617965 17073 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:52:24.617986 17073 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:52:24.619163 17073 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:52:24.619204 17073 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:52:24.619230 17073 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.1
I20260808 16:52:24.620658 17073 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:38273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786207919384338-8288-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786207919384338-8288-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.1.keytab
--principal=kudu/127.8.24.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786207919384338-8288-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.24.1
--webserver_port=0
--tserver_master_addrs=127.8.24.62:38071
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17073
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.1
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786207919384338-8288-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:52:24.620867 17073 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:52:24.621071 17073 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:52:24.625780 17082 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:24.625785 17080 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:24.625921 17073 server_base.cc:1034] running on GCE node
W20260808 16:52:24.626022 17085 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:24.626202 17073 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:52:24.626483 17073 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:52:24.627611 17073 hybrid_clock.cc:648] HybridClock initialized: now 1786207944627601 us; error 28 us; skew 500 ppm
Aug 08 16:52:24 dist-test-slave-6zbq krb5kdc[16893](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207944, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:52:24.629822 17073 init.cc:377] Logged in from keytab as kudu/127.8.24.1@KRBTEST.COM (short username kudu)
I20260808 16:52:24.630208 17073 webserver.cc:460] Webserver started at http://127.8.24.1:34521/ using document root <none> and password file <none>
I20260808 16:52:24.630431 17073 fs_manager.cc:359] Metadata directory not provided
I20260808 16:52:24.630481 17073 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:52:24.630600 17073 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:52:24.631577 17073 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786207919384338-8288-0/minicluster-data/ts-0/data/instance:
uuid: "e3900f270a824cb4a46f489c262be331"
format_stamp: "Formatted at 2026-08-08 16:52:24 on dist-test-slave-6zbq"
server_key: "6021f2252b62660fb902c9def1aeb12c"
server_key_iv: "72c5fce4dc910157c9567aeb700b3aa5"
server_key_version: "encryptionkey@0"
I20260808 16:52:24.631903 17073 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786207919384338-8288-0/minicluster-data/ts-0/wal/instance:
uuid: "e3900f270a824cb4a46f489c262be331"
format_stamp: "Formatted at 2026-08-08 16:52:24 on dist-test-slave-6zbq"
server_key: "6021f2252b62660fb902c9def1aeb12c"
server_key_iv: "72c5fce4dc910157c9567aeb700b3aa5"
server_key_version: "encryptionkey@0"
I20260808 16:52:24.633071 17073 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 16:52:24.633665 17093 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:24.633812 17073 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 16:52:24.633893 17073 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786207919384338-8288-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786207919384338-8288-0/minicluster-data/ts-0/wal
uuid: "e3900f270a824cb4a46f489c262be331"
format_stamp: "Formatted at 2026-08-08 16:52:24 on dist-test-slave-6zbq"
server_key: "6021f2252b62660fb902c9def1aeb12c"
server_key_iv: "72c5fce4dc910157c9567aeb700b3aa5"
server_key_version: "encryptionkey@0"
I20260808 16:52:24.633953 17073 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786207919384338-8288-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786207919384338-8288-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786207919384338-8288-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:52:24.665911 17073 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:52:24.667622 17073 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:52:24.667750 17073 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:52:24.667989 17073 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:52:24.668275 17073 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:52:24.668310 17073 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:24.668339 17073 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:52:24.668356 17073 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:24.673754 17073 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.1:34315
I20260808 16:52:24.673810 17278 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.1:34315 every 8 connection(s)
I20260808 16:52:24.674083 17073 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786207919384338-8288-0/minicluster-data/ts-0/data/info.pb
I20260808 16:52:24.678169 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 17073
I20260808 16:52:24.678293 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786207919384338-8288-0/minicluster-data/ts-0/wal/instance
I20260808 16:52:24.678515 8288 external_mini_cluster.cc:1428] Setting key 4a0bd80f01484c259328e3f4db849b06
Aug 08 16:52:24 dist-test-slave-6zbq krb5kdc[16893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207944, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.1@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260808 16:52:24.682724 17042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:24.674754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:46795 (local address 127.8.24.62:38071)
0808 16:52:24.674814 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:24.674817 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:24.674983 (+ 166us) server_negotiation.cc:407] Connection header received
0808 16:52:24.675834 (+ 851us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:24.675835 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:24.675870 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:24.675927 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:24.676452 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:24.676941 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:24.677817 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:24.677997 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:24.678039 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:24.680220 (+ 2181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:24.680222 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:24.680222 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:24.680229 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:24.681451 (+ 1222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:24.681827 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:24.681828 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:24.681828 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:24.681889 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:24.682146 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:24.682146 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:24.682146 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:24.682240 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:24.682289 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:24.682593 (+ 304us) server_negotiation.cc:299] Negotiation successful
0808 16:52:24.682662 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 16:52:24.682917 17281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:24.674710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:38071 (local address 127.8.24.1:46795)
0808 16:52:24.674866 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:24.674905 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:24.675753 (+ 848us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:24.675976 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:24.675979 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:24.676210 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:24.676387 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:24.676391 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:24.676999 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:24.677000 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:24.677755 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:24.677758 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:24.678076 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:24.678078 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:24.678216 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:24.678686 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:24.678696 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:24.680113 (+ 1417us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:52:24.681529 (+ 1416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:24.681536 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:24.681538 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:24.681747 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:24.681956 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:24.681957 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:24.681957 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:24.682085 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:24.682289 (+ 204us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:52:24.682294 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:52:24.682467 (+ 173us) client_negotiation.cc:769] Sending connection context
0808 16:52:24.682567 (+ 100us) client_negotiation.cc:240] Negotiation successful
0808 16:52:24.682715 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":88,"threads_started":1}
I20260808 16:52:24.683451 17279 heartbeater.cc:347] Connected to a master server at 127.8.24.62:38071
I20260808 16:52:24.683532 17279 heartbeater.cc:464] Registering TS with master...
I20260808 16:52:24.683689 17279 heartbeater.cc:511] Master 127.8.24.62:38071 requested a full tablet report, sending...
I20260808 16:52:24.684197 16973 ts_manager.cc:194] Registered new tserver with Master: e3900f270a824cb4a46f489c262be331 (127.8.24.1:34315)
I20260808 16:52:24.684794 16973 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.1@KRBTEST.COM'} at 127.8.24.1:46795
WARNING: no policy specified for kudu/127.8.24.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.2.keytab.
Entry for principal kudu/127.8.24.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.2.keytab.
I20260808 16:52:24.702183 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786207919384338-8288-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786207919384338-8288-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786207919384338-8288-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786207919384338-8288-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.2:0
--local_ip_for_outbound_sockets=127.8.24.2
--webserver_interface=127.8.24.2
--webserver_port=0
--tserver_master_addrs=127.8.24.62:38071
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.2.keytab
--principal=kudu/127.8.24.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:38273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:52:24.780273 17286 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:52:24.780458 17286 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:52:24.780488 17286 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:52:24.781610 17286 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:52:24.781658 17286 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:52:24.781694 17286 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.2
I20260808 16:52:24.783171 17286 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:38273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786207919384338-8288-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786207919384338-8288-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.2.keytab
--principal=kudu/127.8.24.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786207919384338-8288-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.24.2
--webserver_port=0
--tserver_master_addrs=127.8.24.62:38071
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17286
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.2
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786207919384338-8288-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:52:24.783396 17286 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:52:24.783586 17286 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:52:24.788117 17296 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:24.788117 17293 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:24.788190 17286 server_base.cc:1034] running on GCE node
W20260808 16:52:24.788141 17292 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:24.788524 17286 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:52:24.788740 17286 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:52:24.789881 17286 hybrid_clock.cc:648] HybridClock initialized: now 1786207944789866 us; error 30 us; skew 500 ppm
Aug 08 16:52:24 dist-test-slave-6zbq krb5kdc[16893](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207944, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:52:24.792189 17286 init.cc:377] Logged in from keytab as kudu/127.8.24.2@KRBTEST.COM (short username kudu)
I20260808 16:52:24.792595 17286 webserver.cc:460] Webserver started at http://127.8.24.2:43033/ using document root <none> and password file <none>
I20260808 16:52:24.792788 17286 fs_manager.cc:359] Metadata directory not provided
I20260808 16:52:24.792835 17286 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:52:24.792955 17286 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:52:24.793884 17286 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786207919384338-8288-0/minicluster-data/ts-1/data/instance:
uuid: "2a19df96112d457baac2cdcb0b611994"
format_stamp: "Formatted at 2026-08-08 16:52:24 on dist-test-slave-6zbq"
server_key: "b94927841be45ae1829db6cffc062ecf"
server_key_iv: "97e2d994de72349b0158297b05952855"
server_key_version: "encryptionkey@0"
I20260808 16:52:24.794207 17286 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786207919384338-8288-0/minicluster-data/ts-1/wal/instance:
uuid: "2a19df96112d457baac2cdcb0b611994"
format_stamp: "Formatted at 2026-08-08 16:52:24 on dist-test-slave-6zbq"
server_key: "b94927841be45ae1829db6cffc062ecf"
server_key_iv: "97e2d994de72349b0158297b05952855"
server_key_version: "encryptionkey@0"
I20260808 16:52:24.795512 17286 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 16:52:24.796288 17306 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:24.796463 17286 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 16:52:24.796545 17286 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786207919384338-8288-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786207919384338-8288-0/minicluster-data/ts-1/wal
uuid: "2a19df96112d457baac2cdcb0b611994"
format_stamp: "Formatted at 2026-08-08 16:52:24 on dist-test-slave-6zbq"
server_key: "b94927841be45ae1829db6cffc062ecf"
server_key_iv: "97e2d994de72349b0158297b05952855"
server_key_version: "encryptionkey@0"
I20260808 16:52:24.796607 17286 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786207919384338-8288-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786207919384338-8288-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786207919384338-8288-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:52:24.805217 17286 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:52:24.807089 17286 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:52:24.807205 17286 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:52:24.807394 17286 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:52:24.807651 17286 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:52:24.807689 17286 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:24.807708 17286 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:52:24.807731 17286 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:24.812829 17286 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.2:41601
I20260808 16:52:24.812863 17503 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.2:41601 every 8 connection(s)
I20260808 16:52:24.813126 17286 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786207919384338-8288-0/minicluster-data/ts-1/data/info.pb
I20260808 16:52:24.816247 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 17286
I20260808 16:52:24.816340 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786207919384338-8288-0/minicluster-data/ts-1/wal/instance
I20260808 16:52:24.816490 8288 external_mini_cluster.cc:1428] Setting key 93630dae31ce70cba8b79ce5d62c04e5
Aug 08 16:52:24 dist-test-slave-6zbq krb5kdc[16893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207944, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.2@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260808 16:52:24.821919 17042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:24.813746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:39509 (local address 127.8.24.62:38071)
0808 16:52:24.813796 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:24.813798 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:24.813948 (+ 150us) server_negotiation.cc:407] Connection header received
0808 16:52:24.814847 (+ 899us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:24.814849 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:24.814863 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:24.814914 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:24.815467 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:24.815947 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:24.817004 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:24.817253 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:24.817308 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:24.819313 (+ 2005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:24.819315 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:24.819315 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:24.819322 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:24.820615 (+ 1293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:24.820971 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:24.820972 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:24.820973 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:24.821035 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:24.821273 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:24.821274 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:24.821274 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:24.821407 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:24.821478 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:24.821746 (+ 268us) server_negotiation.cc:299] Negotiation successful
0808 16:52:24.821826 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 16:52:24.821966 17508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:24.813695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:38071 (local address 127.8.24.2:39509)
0808 16:52:24.813837 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:24.813877 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:24.814747 (+ 870us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:24.814975 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:24.814979 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:24.815209 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:24.815376 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:24.815380 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:24.816024 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:24.816025 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:24.816932 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:24.816935 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:24.817326 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:24.817327 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:24.817446 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:24.817841 (+ 395us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:24.817851 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:24.819224 (+ 1373us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:52:24.820700 (+ 1476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:24.820706 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:24.820707 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:24.820907 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:24.821105 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:24.821106 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:24.821106 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:24.821204 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:24.821487 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:52:24.821493 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:52:24.821616 (+ 123us) client_negotiation.cc:769] Sending connection context
0808 16:52:24.821706 (+ 90us) client_negotiation.cc:240] Negotiation successful
0808 16:52:24.821816 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":69,"threads_started":1}
I20260808 16:52:24.822407 17504 heartbeater.cc:347] Connected to a master server at 127.8.24.62:38071
I20260808 16:52:24.822499 17504 heartbeater.cc:464] Registering TS with master...
I20260808 16:52:24.822672 17504 heartbeater.cc:511] Master 127.8.24.62:38071 requested a full tablet report, sending...
I20260808 16:52:24.823040 16973 ts_manager.cc:194] Registered new tserver with Master: 2a19df96112d457baac2cdcb0b611994 (127.8.24.2:41601)
I20260808 16:52:24.823401 16973 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.2@KRBTEST.COM'} at 127.8.24.2:39509
WARNING: no policy specified for kudu/127.8.24.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.3.keytab.
Entry for principal kudu/127.8.24.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.3.keytab.
I20260808 16:52:24.840204 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786207919384338-8288-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786207919384338-8288-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786207919384338-8288-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786207919384338-8288-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.3:0
--local_ip_for_outbound_sockets=127.8.24.3
--webserver_interface=127.8.24.3
--webserver_port=0
--tserver_master_addrs=127.8.24.62:38071
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.3.keytab
--principal=kudu/127.8.24.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:38273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:52:24.917543 17513 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:52:24.917713 17513 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:52:24.917735 17513 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:52:24.918854 17513 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:52:24.918892 17513 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:52:24.918920 17513 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.3
I20260808 16:52:24.920329 17513 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:38273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786207919384338-8288-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786207919384338-8288-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.3.keytab
--principal=kudu/127.8.24.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786207919384338-8288-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.24.3
--webserver_port=0
--tserver_master_addrs=127.8.24.62:38071
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17513
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.3
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786207919384338-8288-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:52:24.920527 17513 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:52:24.920720 17513 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:52:24.925137 17529 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:24.925277 17513 server_base.cc:1034] running on GCE node
W20260808 16:52:24.925161 17523 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:24.925143 17522 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:24.925522 17513 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:52:24.925745 17513 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:52:24.926874 17513 hybrid_clock.cc:648] HybridClock initialized: now 1786207944926858 us; error 28 us; skew 500 ppm
Aug 08 16:52:24 dist-test-slave-6zbq krb5kdc[16893](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207944, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:52:24.929006 17513 init.cc:377] Logged in from keytab as kudu/127.8.24.3@KRBTEST.COM (short username kudu)
I20260808 16:52:24.929384 17513 webserver.cc:460] Webserver started at http://127.8.24.3:39873/ using document root <none> and password file <none>
I20260808 16:52:24.929571 17513 fs_manager.cc:359] Metadata directory not provided
I20260808 16:52:24.929616 17513 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:52:24.929738 17513 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:52:24.930711 17513 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786207919384338-8288-0/minicluster-data/ts-2/data/instance:
uuid: "d798c054847045b08e60379292c2942e"
format_stamp: "Formatted at 2026-08-08 16:52:24 on dist-test-slave-6zbq"
server_key: "d679b513def5c4dbae97364967b303b7"
server_key_iv: "d09c19c59679810d6d6f05a7a7310a49"
server_key_version: "encryptionkey@0"
I20260808 16:52:24.931015 17513 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786207919384338-8288-0/minicluster-data/ts-2/wal/instance:
uuid: "d798c054847045b08e60379292c2942e"
format_stamp: "Formatted at 2026-08-08 16:52:24 on dist-test-slave-6zbq"
server_key: "d679b513def5c4dbae97364967b303b7"
server_key_iv: "d09c19c59679810d6d6f05a7a7310a49"
server_key_version: "encryptionkey@0"
I20260808 16:52:24.932169 17513 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 16:52:24.932752 17543 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:24.932898 17513 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 16:52:24.932976 17513 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786207919384338-8288-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786207919384338-8288-0/minicluster-data/ts-2/wal
uuid: "d798c054847045b08e60379292c2942e"
format_stamp: "Formatted at 2026-08-08 16:52:24 on dist-test-slave-6zbq"
server_key: "d679b513def5c4dbae97364967b303b7"
server_key_iv: "d09c19c59679810d6d6f05a7a7310a49"
server_key_version: "encryptionkey@0"
I20260808 16:52:24.933032 17513 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786207919384338-8288-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786207919384338-8288-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786207919384338-8288-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:52:24.941409 17513 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:52:24.943014 17513 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:52:24.943133 17513 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:52:24.943323 17513 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:52:24.943572 17513 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:52:24.943604 17513 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:24.943629 17513 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:52:24.943647 17513 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:24.949028 17513 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.3:41597
I20260808 16:52:24.949084 17739 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.3:41597 every 8 connection(s)
I20260808 16:52:24.949331 17513 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786207919384338-8288-0/minicluster-data/ts-2/data/info.pb
I20260808 16:52:24.954308 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 17513
I20260808 16:52:24.954422 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786207919384338-8288-0/minicluster-data/ts-2/wal/instance
I20260808 16:52:24.954612 8288 external_mini_cluster.cc:1428] Setting key fc539f39f4dfeef184bd1c634d99299d
Aug 08 16:52:24 dist-test-slave-6zbq krb5kdc[16893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207944, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.3@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260808 16:52:24.957571 17042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:24.949960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:53861 (local address 127.8.24.62:38071)
0808 16:52:24.949999 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:24.950001 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:24.950235 (+ 234us) server_negotiation.cc:407] Connection header received
0808 16:52:24.951051 (+ 816us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:24.951052 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:24.951063 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:24.951117 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:24.951608 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:24.952095 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:24.952971 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:24.953169 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:24.953228 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:24.955248 (+ 2020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:24.955249 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:24.955249 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:24.955256 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:24.956485 (+ 1229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:24.956864 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:24.956865 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:24.956865 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:24.956910 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:24.957073 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:24.957073 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:24.957073 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:24.957160 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:24.957204 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:24.957457 (+ 253us) server_negotiation.cc:299] Negotiation successful
0808 16:52:24.957511 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260808 16:52:24.957747 17743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:24.949957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:38071 (local address 127.8.24.3:53861)
0808 16:52:24.950092 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:24.950122 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:24.950970 (+ 848us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:24.951171 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:24.951174 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:24.951385 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:24.951548 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:24.951552 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:24.952154 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:24.952165 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:24.952889 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:24.952891 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:24.953233 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:24.953234 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:24.953322 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:24.953701 (+ 379us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:24.953708 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:24.955164 (+ 1456us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:52:24.956569 (+ 1405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:24.956576 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:24.956577 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:24.956799 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:24.956949 (+ 150us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:24.956949 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:24.956950 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:24.957033 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:24.957235 (+ 202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:52:24.957241 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:52:24.957351 (+ 110us) client_negotiation.cc:769] Sending connection context
0808 16:52:24.957443 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 16:52:24.957554 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":77,"threads_started":1}
I20260808 16:52:24.958153 17740 heartbeater.cc:347] Connected to a master server at 127.8.24.62:38071
I20260808 16:52:24.958235 17740 heartbeater.cc:464] Registering TS with master...
I20260808 16:52:24.958415 17740 heartbeater.cc:511] Master 127.8.24.62:38071 requested a full tablet report, sending...
I20260808 16:52:24.958736 16973 ts_manager.cc:194] Registered new tserver with Master: d798c054847045b08e60379292c2942e (127.8.24.3:41597)
I20260808 16:52:24.959071 16973 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.3@KRBTEST.COM'} at 127.8.24.3:53861
I20260808 16:52:24.966193 8288 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786207919384338-8288-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786207919384338-8288-0/krb5kdc/test-admin.keytab.
Aug 08 16:52:24 dist-test-slave-6zbq krb5kdc[16893](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207944, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:52:24.981490 8288 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260808 16:52:24.982847 17749 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:52:24.982210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48580 (local address 127.8.24.1:34315)
0808 16:52:24.982388 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:24.982393 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:24.982403 (+ 10us) server_negotiation.cc:407] Connection header received
0808 16:52:24.982424 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:24.982425 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:24.982438 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:24.982501 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:24.982782 (+ 281us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48580: BlockingRecv error: recv got EOF from 127.0.0.1:48580 (error 108)
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":84,"threads_started":1}
W20260808 16:52:24.982930 8288 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786207919384338-8288-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786207919384338-8288-0/krb5kdc/krb5cc)
Aug 08 16:52:24 dist-test-slave-6zbq krb5kdc[16893](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207944, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:52:24.983798 8288 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 16:52:24 dist-test-slave-6zbq krb5kdc[16893](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207944, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:52:24.984668 8288 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 16:52:24 dist-test-slave-6zbq krb5kdc[16893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207944, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.24.1@KRBTEST.COM
I20260808 16:52:24.991459 17749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:24.985149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48594 (local address 127.8.24.1:34315)
0808 16:52:24.985195 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:24.985197 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:24.985224 (+ 27us) server_negotiation.cc:407] Connection header received
0808 16:52:24.985291 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:24.985292 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:24.985303 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:24.985346 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:24.985704 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:24.986434 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:24.987097 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:24.987398 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:24.987456 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:24.989229 (+ 1773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:24.989237 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:24.989238 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:24.989244 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:24.990437 (+ 1193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:24.990743 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:24.990746 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:24.990747 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:24.990800 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:24.990959 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:24.990960 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:24.990960 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:24.991079 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:24.991204 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:24.991294 (+ 90us) server_negotiation.cc:299] Negotiation successful
0808 16:52:24.991363 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
W20260808 16:52:24.992808 17749 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:52:24.992426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48602 (local address 127.8.24.1:34315)
0808 16:52:24.992456 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:24.992458 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:24.992494 (+ 36us) server_negotiation.cc:407] Connection header received
0808 16:52:24.992536 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:24.992536 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:24.992543 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:24.992586 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:24.992778 (+ 192us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48602: BlockingRecv error: recv got EOF from 127.0.0.1:48602 (error 108)
Metrics: {"server-negotiator.queue_time_us":14}
W20260808 16:52:24.992964 8288 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786207919384338-8288-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786207919384338-8288-0/krb5kdc/krb5cc)
Aug 08 16:52:24 dist-test-slave-6zbq krb5kdc[16893](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207944, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:52:24.993777 8288 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 16:52:24 dist-test-slave-6zbq krb5kdc[16893](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207944, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:52:24.994525 8288 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 16:52:24 dist-test-slave-6zbq krb5kdc[16893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207944, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.24.1@KRBTEST.COM
I20260808 16:52:25.000337 17749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:24.994932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48604 (local address 127.8.24.1:34315)
0808 16:52:24.994968 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:24.994970 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:24.994998 (+ 28us) server_negotiation.cc:407] Connection header received
0808 16:52:24.995042 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:24.995043 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:24.995050 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:24.995094 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:24.995376 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:24.995813 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:24.996410 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:24.996580 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:24.996622 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:24.998290 (+ 1668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:24.998291 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:24.998291 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:24.998297 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:24.999384 (+ 1087us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:24.999776 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:24.999777 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:24.999777 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:24.999822 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:24.999955 (+ 133us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:24.999955 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:24.999955 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:25.000026 (+ 71us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:25.000105 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:25.000230 (+ 125us) server_negotiation.cc:299] Negotiation successful
0808 16:52:25.000288 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
W20260808 16:52:25.001612 17749 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:52:25.001136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48606 (local address 127.8.24.1:34315)
0808 16:52:25.001178 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:25.001180 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:25.001222 (+ 42us) server_negotiation.cc:407] Connection header received
0808 16:52:25.001286 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:25.001286 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:25.001294 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:25.001333 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:25.001576 (+ 243us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48606: BlockingRecv error: recv got EOF from 127.0.0.1:48606 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
W20260808 16:52:25.001662 8288 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786207919384338-8288-0/krb5kdc/krb5cc' of type FILE
Aug 08 16:52:24 dist-test-slave-6zbq krb5kdc[16893](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207944, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:52:25.002486 8288 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 16:52:24 dist-test-slave-6zbq krb5kdc[16893](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207944, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:52:25.003219 8288 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 16:52:25 dist-test-slave-6zbq krb5kdc[16893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207944, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.24.1@KRBTEST.COM
I20260808 16:52:25.009074 17749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:25.003648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48620 (local address 127.8.24.1:34315)
0808 16:52:25.003687 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:25.003689 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:25.003763 (+ 74us) server_negotiation.cc:407] Connection header received
0808 16:52:25.003815 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:25.003816 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:25.003824 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:25.003869 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:25.004187 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:25.004606 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:25.005230 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:25.005402 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:25.005453 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:25.007179 (+ 1726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:25.007180 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:25.007181 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:25.007187 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:25.008184 (+ 997us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:25.008460 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:25.008461 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:25.008461 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:25.008507 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:25.008672 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:25.008672 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:25.008673 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:25.008752 (+ 79us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:25.008824 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:25.008929 (+ 105us) server_negotiation.cc:299] Negotiation successful
0808 16:52:25.008996 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 16:52:25.009572 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 17073
I20260808 16:52:25.025863 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 17286
I20260808 16:52:25.035838 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 17513
I20260808 16:52:25.043186 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 16919
2026-08-08T16:52:25Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (1917 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 16:52:25 dist-test-slave-6zbq krb5kdc[17768](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 16:52:25 dist-test-slave-6zbq krb5kdc[17768](info): set up 2 sockets
Aug 08 16:52:25 dist-test-slave-6zbq krb5kdc[17768](info): commencing operation
krb5kdc: starting...
W20260808 16:52:26.251741 8288 mini_kdc.cc:121] Time spent starting KDC: real 1.187s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
W20260808 16:52:26.500983 8288 mini_kdc.cc:253] Time spent creating user principal test-admin: real 0.249s user 0.000s sys 0.002s
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
W20260808 16:52:26.720139 8288 mini_kdc.cc:253] Time spent creating user principal test-user: real 0.219s user 0.000s sys 0.002s
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
W20260808 16:52:26.878295 8288 mini_kdc.cc:253] Time spent creating user principal joe-interloper: real 0.158s user 0.000s sys 0.002s
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/krb5kdc/test-user.keytab.
W20260808 16:52:27.125169 8288 mini_kdc.cc:302] Time spent creating keytab for test-user: real 0.247s user 0.002s sys 0.000s
Aug 08 16:52:27 dist-test-slave-6zbq krb5kdc[17768](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207947, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T16:52:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T16:52:27Z Disabled control of system clock
WARNING: no policy specified for oryx/127.8.24.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.24.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.24.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/krb5kdc/oryx_127.8.24.62.keytab.
Entry for principal oryx/127.8.24.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/krb5kdc/oryx_127.8.24.62.keytab.
I20260808 16:52:27.164647 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:46193
--webserver_interface=127.8.24.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/krb5kdc/oryx_127.8.24.62.keytab
--principal=oryx/127.8.24.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:37767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:46193
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:52:27.243065 17791 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:52:27.243252 17791 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:52:27.243283 17791 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:52:27.244357 17791 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 16:52:27.244395 17791 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:52:27.244412 17791 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:52:27.244427 17791 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 16:52:27.244444 17791 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 16:52:27.245903 17791 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:37767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:46193
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/krb5kdc/oryx_127.8.24.62.keytab
--principal=oryx/127.8.24.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:46193
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.24.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17791
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:52:27.246135 17791 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:52:27.246328 17791 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:52:27.251374 17802 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:27.251386 17799 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:27.251402 17800 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:27.251734 17791 server_base.cc:1034] running on GCE node
I20260808 16:52:27.251988 17791 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:52:27.252311 17791 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:52:27.253505 17791 hybrid_clock.cc:648] HybridClock initialized: now 1786207947253488 us; error 32 us; skew 500 ppm
Aug 08 16:52:27 dist-test-slave-6zbq krb5kdc[17768](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207947, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.24.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:52:27.260248 17791 init.cc:377] Logged in from keytab as oryx/127.8.24.62@KRBTEST.COM (short username oryx)
I20260808 16:52:27.260712 17791 webserver.cc:460] Webserver started at http://127.8.24.62:38283/ using document root <none> and password file <none>
I20260808 16:52:27.260938 17791 fs_manager.cc:359] Metadata directory not provided
I20260808 16:52:27.260987 17791 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:52:27.261129 17791 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:52:27.262195 17791 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/master-0/data/instance:
uuid: "678b35993e9b49aca0f3ad2bba5edfbd"
format_stamp: "Formatted at 2026-08-08 16:52:27 on dist-test-slave-6zbq"
server_key: "8ff2bef720969fe19b2ebc53946fc6de"
server_key_iv: "567a4ada82c3b338f686a37bc0b700ea"
server_key_version: "encryptionkey@0"
I20260808 16:52:27.262598 17791 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/master-0/wal/instance:
uuid: "678b35993e9b49aca0f3ad2bba5edfbd"
format_stamp: "Formatted at 2026-08-08 16:52:27 on dist-test-slave-6zbq"
server_key: "8ff2bef720969fe19b2ebc53946fc6de"
server_key_iv: "567a4ada82c3b338f686a37bc0b700ea"
server_key_version: "encryptionkey@0"
I20260808 16:52:27.263855 17791 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 16:52:27.264526 17816 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:27.264694 17791 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 16:52:27.264783 17791 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/master-0/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/master-0/wal
uuid: "678b35993e9b49aca0f3ad2bba5edfbd"
format_stamp: "Formatted at 2026-08-08 16:52:27 on dist-test-slave-6zbq"
server_key: "8ff2bef720969fe19b2ebc53946fc6de"
server_key_iv: "567a4ada82c3b338f686a37bc0b700ea"
server_key_version: "encryptionkey@0"
I20260808 16:52:27.264850 17791 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:52:27.286669 17791 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:52:27.288925 17791 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:52:27.289076 17791 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:52:27.293038 17791 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.62:46193
I20260808 16:52:27.293080 17906 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.62:46193 every 8 connection(s)
I20260808 16:52:27.293537 17791 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
I20260808 16:52:27.294560 17907 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:52:27.296799 17907 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 678b35993e9b49aca0f3ad2bba5edfbd: Bootstrap starting.
I20260808 16:52:27.297361 17907 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 678b35993e9b49aca0f3ad2bba5edfbd: Neither blocks nor log segments found. Creating new log.
I20260808 16:52:27.297614 17907 log.cc:824] T 00000000000000000000000000000000 P 678b35993e9b49aca0f3ad2bba5edfbd: Log is configured to *not* fsync() on all Append() calls
I20260808 16:52:27.298283 17907 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 678b35993e9b49aca0f3ad2bba5edfbd: No bootstrap required, opened a new log
I20260808 16:52:27.298852 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 17791
I20260808 16:52:27.298945 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/master-0/wal/instance
I20260808 16:52:27.299113 8288 external_mini_cluster.cc:1428] Setting key a5d894dd0abcb5cbb1049679be45ecf4
I20260808 16:52:27.300235 17907 raft_consensus.cc:348] T 00000000000000000000000000000000 P 678b35993e9b49aca0f3ad2bba5edfbd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "678b35993e9b49aca0f3ad2bba5edfbd" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 46193 } }
I20260808 16:52:27.300406 17907 raft_consensus.cc:374] T 00000000000000000000000000000000 P 678b35993e9b49aca0f3ad2bba5edfbd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:52:27.300443 17907 raft_consensus.cc:729] T 00000000000000000000000000000000 P 678b35993e9b49aca0f3ad2bba5edfbd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 678b35993e9b49aca0f3ad2bba5edfbd, State: Initialized, Role: FOLLOWER
I20260808 16:52:27.300595 17907 consensus_queue.cc:260] T 00000000000000000000000000000000 P 678b35993e9b49aca0f3ad2bba5edfbd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "678b35993e9b49aca0f3ad2bba5edfbd" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 46193 } }
I20260808 16:52:27.300679 17907 raft_consensus.cc:388] T 00000000000000000000000000000000 P 678b35993e9b49aca0f3ad2bba5edfbd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 16:52:27.300702 17907 raft_consensus.cc:482] T 00000000000000000000000000000000 P 678b35993e9b49aca0f3ad2bba5edfbd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 16:52:27.300747 17907 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 678b35993e9b49aca0f3ad2bba5edfbd [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:52:27.301867 17907 raft_consensus.cc:504] T 00000000000000000000000000000000 P 678b35993e9b49aca0f3ad2bba5edfbd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "678b35993e9b49aca0f3ad2bba5edfbd" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 46193 } }
I20260808 16:52:27.301975 17907 leader_election.cc:302] T 00000000000000000000000000000000 P 678b35993e9b49aca0f3ad2bba5edfbd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 678b35993e9b49aca0f3ad2bba5edfbd; no voters:
I20260808 16:52:27.302115 17907 leader_election.cc:288] T 00000000000000000000000000000000 P 678b35993e9b49aca0f3ad2bba5edfbd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 16:52:27.302156 17919 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 678b35993e9b49aca0f3ad2bba5edfbd [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:52:27.302304 17919 raft_consensus.cc:686] T 00000000000000000000000000000000 P 678b35993e9b49aca0f3ad2bba5edfbd [term 1 LEADER]: Becoming Leader. State: Replica: 678b35993e9b49aca0f3ad2bba5edfbd, State: Running, Role: LEADER
I20260808 16:52:27.302358 17907 sys_catalog.cc:564] T 00000000000000000000000000000000 P 678b35993e9b49aca0f3ad2bba5edfbd [sys.catalog]: configured and running, proceeding with master startup.
I20260808 16:52:27.302461 17919 consensus_queue.cc:237] T 00000000000000000000000000000000 P 678b35993e9b49aca0f3ad2bba5edfbd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "678b35993e9b49aca0f3ad2bba5edfbd" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 46193 } }
I20260808 16:52:27.302886 17920 sys_catalog.cc:455] T 00000000000000000000000000000000 P 678b35993e9b49aca0f3ad2bba5edfbd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "678b35993e9b49aca0f3ad2bba5edfbd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "678b35993e9b49aca0f3ad2bba5edfbd" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 46193 } } }
I20260808 16:52:27.302971 17920 sys_catalog.cc:458] T 00000000000000000000000000000000 P 678b35993e9b49aca0f3ad2bba5edfbd [sys.catalog]: This master's current role is: LEADER
I20260808 16:52:27.303167 17921 sys_catalog.cc:455] T 00000000000000000000000000000000 P 678b35993e9b49aca0f3ad2bba5edfbd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 678b35993e9b49aca0f3ad2bba5edfbd. Latest consensus state: current_term: 1 leader_uuid: "678b35993e9b49aca0f3ad2bba5edfbd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "678b35993e9b49aca0f3ad2bba5edfbd" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 46193 } } }
I20260808 16:52:27.303268 17921 sys_catalog.cc:458] T 00000000000000000000000000000000 P 678b35993e9b49aca0f3ad2bba5edfbd [sys.catalog]: This master's current role is: LEADER
I20260808 16:52:27.303438 17931 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 16:52:27.303964 17931 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 16:52:27.305570 17931 catalog_manager.cc:1347] Generated new cluster ID: ddde3755643b4afbb76aa4f0f8896bc8
I20260808 16:52:27.305621 17931 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 16:52:27 dist-test-slave-6zbq krb5kdc[17768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207947, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM
I20260808 16:52:27.310683 17917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:27.299619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58964 (local address 127.8.24.62:46193)
0808 16:52:27.299839 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:27.299849 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:27.299885 (+ 36us) server_negotiation.cc:407] Connection header received
0808 16:52:27.300205 (+ 320us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:27.300213 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:27.300276 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:27.300421 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:27.301074 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:27.302480 (+ 1406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:27.303479 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:27.304050 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:27.304173 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:27.307003 (+ 2830us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:27.307008 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:27.307008 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:27.307020 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:27.309060 (+ 2040us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:27.309426 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:27.309429 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:27.309429 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:27.309557 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:27.309737 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:27.309738 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:27.309738 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:27.310069 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:27.310244 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:27.310313 (+ 69us) server_negotiation.cc:299] Negotiation successful
0808 16:52:27.310487 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":94,"threads_started":1}
I20260808 16:52:27.326638 17931 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 16:52:27.327138 17931 catalog_manager.cc:1504] Loading token signing keys...
I20260808 16:52:27.334868 17931 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 678b35993e9b49aca0f3ad2bba5edfbd: Generated new TSK 0
I20260808 16:52:27.335021 17931 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.8.24.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.24.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.24.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/krb5kdc/oryx_127.8.24.1.keytab.
Entry for principal oryx/127.8.24.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/krb5kdc/oryx_127.8.24.1.keytab.
I20260808 16:52:27.360834 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.1:0
--local_ip_for_outbound_sockets=127.8.24.1
--webserver_interface=127.8.24.1
--webserver_port=0
--tserver_master_addrs=127.8.24.62:46193
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/krb5kdc/oryx_127.8.24.1.keytab
--principal=oryx/127.8.24.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:37767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:52:27.438355 17954 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:52:27.438526 17954 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:52:27.438547 17954 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:52:27.439628 17954 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:52:27.439670 17954 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:52:27.439700 17954 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.1
I20260808 16:52:27.441082 17954 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:37767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/krb5kdc/oryx_127.8.24.1.keytab
--principal=oryx/127.8.24.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.24.1
--webserver_port=0
--tserver_master_addrs=127.8.24.62:46193
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17954
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.1
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:52:27.441305 17954 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:52:27.441504 17954 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:52:27.445912 17969 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:27.445937 17965 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:27.445938 17963 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:27.446246 17954 server_base.cc:1034] running on GCE node
I20260808 16:52:27.446460 17954 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:52:27.446676 17954 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:52:27.447813 17954 hybrid_clock.cc:648] HybridClock initialized: now 1786207947447796 us; error 29 us; skew 500 ppm
Aug 08 16:52:27 dist-test-slave-6zbq krb5kdc[17768](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207947, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.24.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:52:27.450080 17954 init.cc:377] Logged in from keytab as oryx/127.8.24.1@KRBTEST.COM (short username oryx)
I20260808 16:52:27.450506 17954 webserver.cc:460] Webserver started at http://127.8.24.1:33339/ using document root <none> and password file <none>
I20260808 16:52:27.450698 17954 fs_manager.cc:359] Metadata directory not provided
I20260808 16:52:27.450742 17954 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:52:27.450863 17954 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:52:27.451784 17954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/ts-0/data/instance:
uuid: "7b46262135f64e30b03b7b7cfa5e7044"
format_stamp: "Formatted at 2026-08-08 16:52:27 on dist-test-slave-6zbq"
server_key: "bed61d00eef2026f4dbe6727862f0cab"
server_key_iv: "bece949b1fa06c287a8a8e977305c2af"
server_key_version: "encryptionkey@0"
I20260808 16:52:27.452090 17954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/ts-0/wal/instance:
uuid: "7b46262135f64e30b03b7b7cfa5e7044"
format_stamp: "Formatted at 2026-08-08 16:52:27 on dist-test-slave-6zbq"
server_key: "bed61d00eef2026f4dbe6727862f0cab"
server_key_iv: "bece949b1fa06c287a8a8e977305c2af"
server_key_version: "encryptionkey@0"
I20260808 16:52:27.453259 17954 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 16:52:27.453847 17982 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:27.453989 17954 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:27.454069 17954 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/ts-0/wal
uuid: "7b46262135f64e30b03b7b7cfa5e7044"
format_stamp: "Formatted at 2026-08-08 16:52:27 on dist-test-slave-6zbq"
server_key: "bed61d00eef2026f4dbe6727862f0cab"
server_key_iv: "bece949b1fa06c287a8a8e977305c2af"
server_key_version: "encryptionkey@0"
I20260808 16:52:27.454120 17954 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:52:27.469274 17954 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:52:27.470821 17954 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:52:27.470920 17954 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:52:27.471076 17954 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:52:27.471302 17954 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:52:27.471333 17954 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:27.471349 17954 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:52:27.471365 17954 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:27.476321 17954 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.1:38887
I20260808 16:52:27.476382 18178 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.1:38887 every 8 connection(s)
I20260808 16:52:27.476610 17954 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/ts-0/data/info.pb
Aug 08 16:52:27 dist-test-slave-6zbq krb5kdc[17768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207947, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.24.1@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM
I20260808 16:52:27.484513 17917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:27.477178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:50887 (local address 127.8.24.62:46193)
0808 16:52:27.477221 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:27.477223 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:27.477411 (+ 188us) server_negotiation.cc:407] Connection header received
0808 16:52:27.477849 (+ 438us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:27.477851 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:27.477865 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:27.477922 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:27.478859 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:27.479344 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:27.480278 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:27.480479 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:27.480533 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:27.482288 (+ 1755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:27.482289 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:27.482290 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:27.482300 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:27.483350 (+ 1050us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:27.483690 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:27.483691 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:27.483691 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:27.483743 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:27.483945 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:27.483946 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:27.483946 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:27.484051 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:27.484106 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:27.484342 (+ 236us) server_negotiation.cc:299] Negotiation successful
0808 16:52:27.484425 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 16:52:27.484634 18184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:27.477137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:46193 (local address 127.8.24.1:50887)
0808 16:52:27.477313 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:27.477344 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:27.477760 (+ 416us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:27.477987 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:27.477991 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:27.478593 (+ 602us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:27.478772 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:27.478776 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:27.479423 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:27.479424 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:27.480198 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:27.480200 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:27.480547 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:27.480548 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:27.480667 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:27.481006 (+ 339us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:27.481014 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:27.482204 (+ 1190us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:52:27.483421 (+ 1217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:27.483428 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:27.483430 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:27.483625 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:27.483797 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:27.483798 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:27.483798 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:27.483891 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:27.484105 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:52:27.484112 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:52:27.484230 (+ 118us) client_negotiation.cc:769] Sending connection context
0808 16:52:27.484312 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 16:52:27.484449 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":103,"threads_started":1}
I20260808 16:52:27.484900 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 17954
I20260808 16:52:27.484990 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/ts-0/wal/instance
I20260808 16:52:27.485117 18181 heartbeater.cc:347] Connected to a master server at 127.8.24.62:46193
I20260808 16:52:27.485183 8288 external_mini_cluster.cc:1428] Setting key 94fc372ac4d8284567944d0dac052681
I20260808 16:52:27.485208 18181 heartbeater.cc:464] Registering TS with master...
I20260808 16:52:27.485365 18181 heartbeater.cc:511] Master 127.8.24.62:46193 requested a full tablet report, sending...
I20260808 16:52:27.485849 17840 ts_manager.cc:194] Registered new tserver with Master: 7b46262135f64e30b03b7b7cfa5e7044 (127.8.24.1:38887)
I20260808 16:52:27.486645 17840 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.24.1@KRBTEST.COM'} at 127.8.24.1:50887
WARNING: no policy specified for oryx/127.8.24.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.24.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.24.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/krb5kdc/oryx_127.8.24.2.keytab.
Entry for principal oryx/127.8.24.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/krb5kdc/oryx_127.8.24.2.keytab.
I20260808 16:52:27.508438 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.2:0
--local_ip_for_outbound_sockets=127.8.24.2
--webserver_interface=127.8.24.2
--webserver_port=0
--tserver_master_addrs=127.8.24.62:46193
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/krb5kdc/oryx_127.8.24.2.keytab
--principal=oryx/127.8.24.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:37767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:52:27.585754 18187 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:52:27.585933 18187 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:52:27.585963 18187 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:52:27.587065 18187 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:52:27.587105 18187 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:52:27.587149 18187 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.2
I20260808 16:52:27.588541 18187 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:37767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/krb5kdc/oryx_127.8.24.2.keytab
--principal=oryx/127.8.24.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.24.2
--webserver_port=0
--tserver_master_addrs=127.8.24.62:46193
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18187
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.2
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:52:27.588755 18187 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:52:27.588949 18187 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:52:27.593372 18202 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:27.593385 18204 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:27.593437 18201 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:27.593641 18187 server_base.cc:1034] running on GCE node
I20260808 16:52:27.593822 18187 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:52:27.594012 18187 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:52:27.595145 18187 hybrid_clock.cc:648] HybridClock initialized: now 1786207947595134 us; error 28 us; skew 500 ppm
Aug 08 16:52:27 dist-test-slave-6zbq krb5kdc[17768](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207947, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.24.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:52:27.597283 18187 init.cc:377] Logged in from keytab as oryx/127.8.24.2@KRBTEST.COM (short username oryx)
I20260808 16:52:27.597674 18187 webserver.cc:460] Webserver started at http://127.8.24.2:41471/ using document root <none> and password file <none>
I20260808 16:52:27.597864 18187 fs_manager.cc:359] Metadata directory not provided
I20260808 16:52:27.597909 18187 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:52:27.598026 18187 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:52:27.598973 18187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/ts-1/data/instance:
uuid: "005e71fe78a847c2b010b116c89a8609"
format_stamp: "Formatted at 2026-08-08 16:52:27 on dist-test-slave-6zbq"
server_key: "a20a77b1d9a353bcd84c52447d3c3e00"
server_key_iv: "f7bcfc50abde4a3f92452e69b76b9e4b"
server_key_version: "encryptionkey@0"
I20260808 16:52:27.599291 18187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/ts-1/wal/instance:
uuid: "005e71fe78a847c2b010b116c89a8609"
format_stamp: "Formatted at 2026-08-08 16:52:27 on dist-test-slave-6zbq"
server_key: "a20a77b1d9a353bcd84c52447d3c3e00"
server_key_iv: "f7bcfc50abde4a3f92452e69b76b9e4b"
server_key_version: "encryptionkey@0"
I20260808 16:52:27.600432 18187 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 16:52:27.601020 18217 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:27.601189 18187 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 16:52:27.601269 18187 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/ts-1/wal
uuid: "005e71fe78a847c2b010b116c89a8609"
format_stamp: "Formatted at 2026-08-08 16:52:27 on dist-test-slave-6zbq"
server_key: "a20a77b1d9a353bcd84c52447d3c3e00"
server_key_iv: "f7bcfc50abde4a3f92452e69b76b9e4b"
server_key_version: "encryptionkey@0"
I20260808 16:52:27.601325 18187 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:52:27.634835 18187 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:52:27.636327 18187 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:52:27.636447 18187 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:52:27.636626 18187 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:52:27.636880 18187 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:52:27.636911 18187 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:27.636937 18187 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:52:27.636955 18187 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:27.642184 18187 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.2:38759
I20260808 16:52:27.642217 18389 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.2:38759 every 8 connection(s)
I20260808 16:52:27.642488 18187 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/ts-1/data/info.pb
Aug 08 16:52:27 dist-test-slave-6zbq krb5kdc[17768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207947, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.24.2@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM
I20260808 16:52:27.650584 17917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:27.643087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:58627 (local address 127.8.24.62:46193)
0808 16:52:27.643135 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:27.643137 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:27.643317 (+ 180us) server_negotiation.cc:407] Connection header received
0808 16:52:27.643758 (+ 441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:27.643760 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:27.643773 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:27.643819 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:27.644687 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:27.645175 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:27.646152 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:27.646377 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:27.646443 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:27.648331 (+ 1888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:27.648332 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:27.648332 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:27.648340 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:27.649354 (+ 1014us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:27.649705 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:27.649706 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:27.649707 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:27.649761 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:27.649970 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:27.649971 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:27.649971 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:27.650081 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:27.650137 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:27.650410 (+ 273us) server_negotiation.cc:299] Negotiation successful
0808 16:52:27.650495 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 16:52:27.650697 18394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:27.643047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:46193 (local address 127.8.24.2:58627)
0808 16:52:27.643205 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:27.643238 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:27.643659 (+ 421us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:27.643895 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:27.643900 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:27.644420 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:27.644583 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:27.644588 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:27.645254 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:27.645255 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:27.646074 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:27.646078 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:27.646452 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:27.646453 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:27.646575 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:27.646981 (+ 406us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:27.646990 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:27.648248 (+ 1258us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:52:27.649425 (+ 1177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:27.649431 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:27.649433 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:27.649631 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:27.649816 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:27.649816 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:27.649817 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:27.649914 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:27.650134 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:52:27.650140 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:52:27.650270 (+ 130us) client_negotiation.cc:769] Sending connection context
0808 16:52:27.650370 (+ 100us) client_negotiation.cc:240] Negotiation successful
0808 16:52:27.650511 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":78,"threads_started":1}
I20260808 16:52:27.651119 18390 heartbeater.cc:347] Connected to a master server at 127.8.24.62:46193
I20260808 16:52:27.651198 18390 heartbeater.cc:464] Registering TS with master...
I20260808 16:52:27.651350 18390 heartbeater.cc:511] Master 127.8.24.62:46193 requested a full tablet report, sending...
I20260808 16:52:27.651674 17840 ts_manager.cc:194] Registered new tserver with Master: 005e71fe78a847c2b010b116c89a8609 (127.8.24.2:38759)
I20260808 16:52:27.652012 17840 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.24.2@KRBTEST.COM'} at 127.8.24.2:58627
I20260808 16:52:27.652580 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 18187
I20260808 16:52:27.652663 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/ts-1/wal/instance
I20260808 16:52:27.652798 8288 external_mini_cluster.cc:1428] Setting key 88205d9bf3897996f266786e5716142a
WARNING: no policy specified for oryx/127.8.24.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.24.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.24.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/krb5kdc/oryx_127.8.24.3.keytab.
Entry for principal oryx/127.8.24.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/krb5kdc/oryx_127.8.24.3.keytab.
I20260808 16:52:27.674515 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.3:0
--local_ip_for_outbound_sockets=127.8.24.3
--webserver_interface=127.8.24.3
--webserver_port=0
--tserver_master_addrs=127.8.24.62:46193
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/krb5kdc/oryx_127.8.24.3.keytab
--principal=oryx/127.8.24.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:37767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:52:27.751519 18399 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:52:27.751691 18399 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:52:27.751713 18399 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:52:27.752748 18399 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:52:27.752779 18399 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:52:27.752803 18399 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.3
I20260808 16:52:27.754135 18399 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:37767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/krb5kdc/oryx_127.8.24.3.keytab
--principal=oryx/127.8.24.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.24.3
--webserver_port=0
--tserver_master_addrs=127.8.24.62:46193
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18399
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.3
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:52:27.754300 18399 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:52:27.754499 18399 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:52:27.758885 18410 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:27.758962 18408 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:27.758899 18407 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:27.759016 18399 server_base.cc:1034] running on GCE node
I20260808 16:52:27.759253 18399 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:52:27.759469 18399 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:52:27.760597 18399 hybrid_clock.cc:648] HybridClock initialized: now 1786207947760583 us; error 32 us; skew 500 ppm
Aug 08 16:52:27 dist-test-slave-6zbq krb5kdc[17768](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207947, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.24.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:52:27.762899 18399 init.cc:377] Logged in from keytab as oryx/127.8.24.3@KRBTEST.COM (short username oryx)
I20260808 16:52:27.763290 18399 webserver.cc:460] Webserver started at http://127.8.24.3:35665/ using document root <none> and password file <none>
I20260808 16:52:27.763479 18399 fs_manager.cc:359] Metadata directory not provided
I20260808 16:52:27.763520 18399 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:52:27.763602 18399 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:52:27.764465 18399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/ts-2/data/instance:
uuid: "5f52c81c0e744fd195de3a1f27250198"
format_stamp: "Formatted at 2026-08-08 16:52:27 on dist-test-slave-6zbq"
server_key: "4190edc4403864b54992fc64510692f8"
server_key_iv: "095d89c1b6e40fd0bb3a8944d9ba0f20"
server_key_version: "encryptionkey@0"
I20260808 16:52:27.764730 18399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/ts-2/wal/instance:
uuid: "5f52c81c0e744fd195de3a1f27250198"
format_stamp: "Formatted at 2026-08-08 16:52:27 on dist-test-slave-6zbq"
server_key: "4190edc4403864b54992fc64510692f8"
server_key_iv: "095d89c1b6e40fd0bb3a8944d9ba0f20"
server_key_version: "encryptionkey@0"
I20260808 16:52:27.765830 18399 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 16:52:27.766453 18423 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:27.766590 18399 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:27.766664 18399 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/ts-2/wal
uuid: "5f52c81c0e744fd195de3a1f27250198"
format_stamp: "Formatted at 2026-08-08 16:52:27 on dist-test-slave-6zbq"
server_key: "4190edc4403864b54992fc64510692f8"
server_key_iv: "095d89c1b6e40fd0bb3a8944d9ba0f20"
server_key_version: "encryptionkey@0"
I20260808 16:52:27.766731 18399 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:52:27.798689 18399 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:52:27.800248 18399 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:52:27.800364 18399 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:52:27.800549 18399 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:52:27.800812 18399 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:52:27.800843 18399 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:27.800863 18399 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:52:27.800885 18399 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:27.806670 18399 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.3:40919
I20260808 16:52:27.806728 18610 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.3:40919 every 8 connection(s)
I20260808 16:52:27.806984 18399 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/ts-2/data/info.pb
I20260808 16:52:27.808609 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 18399
I20260808 16:52:27.808727 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/ts-2/wal/instance
I20260808 16:52:27.808905 8288 external_mini_cluster.cc:1428] Setting key 6bbac7ee6a124e9f63b8d64e7b2cb8d2
Aug 08 16:52:27 dist-test-slave-6zbq krb5kdc[17768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207947, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.24.3@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM
I20260808 16:52:27.815076 17917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:27.807573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:37091 (local address 127.8.24.62:46193)
0808 16:52:27.807614 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:27.807615 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:27.807789 (+ 174us) server_negotiation.cc:407] Connection header received
0808 16:52:27.808224 (+ 435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:27.808225 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:27.808234 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:27.808283 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:27.809363 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:27.809990 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:27.810977 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:27.811165 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:27.811216 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:27.813011 (+ 1795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:27.813012 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:27.813012 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:27.813018 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:27.814046 (+ 1028us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:27.814408 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:27.814409 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:27.814409 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:27.814455 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:27.814635 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:27.814635 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:27.814636 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:27.814733 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:27.814777 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:27.814962 (+ 185us) server_negotiation.cc:299] Negotiation successful
0808 16:52:27.815017 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 16:52:27.815223 18619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:27.807535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:46193 (local address 127.8.24.3:37091)
0808 16:52:27.807685 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:27.807722 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:27.808143 (+ 421us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:27.808333 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:27.808337 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:27.809014 (+ 677us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:27.809279 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:27.809286 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:27.810059 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:27.810060 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:27.810906 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:27.810909 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:27.811224 (+ 315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:27.811224 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:27.811317 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:27.811719 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:27.811726 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:27.812931 (+ 1205us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:52:27.814106 (+ 1175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:27.814111 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:27.814112 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:27.814314 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:27.814502 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:27.814502 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:27.814503 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:27.814593 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:27.814775 (+ 182us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:52:27.814778 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:52:27.814873 (+ 95us) client_negotiation.cc:769] Sending connection context
0808 16:52:27.814944 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 16:52:27.815048 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":80,"threads_started":1}
I20260808 16:52:27.815627 18613 heartbeater.cc:347] Connected to a master server at 127.8.24.62:46193
I20260808 16:52:27.815718 18613 heartbeater.cc:464] Registering TS with master...
I20260808 16:52:27.815873 18613 heartbeater.cc:511] Master 127.8.24.62:46193 requested a full tablet report, sending...
I20260808 16:52:27.816185 17840 ts_manager.cc:194] Registered new tserver with Master: 5f52c81c0e744fd195de3a1f27250198 (127.8.24.3:40919)
I20260808 16:52:27.816529 17840 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.24.3@KRBTEST.COM'} at 127.8.24.3:37091
I20260808 16:52:27.820318 8288 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.8.24.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.24.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.24.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/krb5kdc/oryx_127.8.24.61.keytab.
Entry for principal oryx/127.8.24.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/krb5kdc/oryx_127.8.24.61.keytab.
I20260808 16:52:27.842555 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.61:45755
--webserver_interface=127.8.24.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/krb5kdc/oryx_127.8.24.61.keytab
--principal=oryx/127.8.24.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:37767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:46193,127.8.24.61:45755
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:52:27.920612 18627 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:52:27.920786 18627 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:52:27.920806 18627 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:52:27.921849 18627 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 16:52:27.921882 18627 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:52:27.921895 18627 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:52:27.921907 18627 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 16:52:27.921921 18627 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 16:52:27.923309 18627 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:37767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:46193,127.8.24.61:45755
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/krb5kdc/oryx_127.8.24.61.keytab
--principal=oryx/127.8.24.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.61:45755
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.8.24.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18627
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
W20260808 16:52:27.923405 18627 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.8.24.62:46193,127.8.24.61:45755'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260808 16:52:27.923501 18627 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:52:27.923651 18627 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 16:52:27.927906 18627 server_base.cc:1034] running on GCE node
W20260808 16:52:27.927852 18636 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:27.927927 18634 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:27.927871 18633 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:27.928306 18627 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:52:27.928553 18627 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:52:27.929685 18627 hybrid_clock.cc:648] HybridClock initialized: now 1786207947929673 us; error 27 us; skew 500 ppm
Aug 08 16:52:27 dist-test-slave-6zbq krb5kdc[17768](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207947, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.24.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:52:27.931834 18627 init.cc:377] Logged in from keytab as oryx/127.8.24.61@KRBTEST.COM (short username oryx)
I20260808 16:52:27.932196 18627 webserver.cc:460] Webserver started at http://127.8.24.61:43755/ using document root <none> and password file <none>
I20260808 16:52:27.932385 18627 fs_manager.cc:359] Metadata directory not provided
I20260808 16:52:27.932430 18627 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:52:27.932546 18627 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:52:27.933482 18627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/master-1/data/instance:
uuid: "ca479471edff409d957dde1bb92c4144"
format_stamp: "Formatted at 2026-08-08 16:52:27 on dist-test-slave-6zbq"
server_key: "fc8a37e8e3c79aa195396dc82db853c2"
server_key_iv: "46ef5e14187bc3bc6f4ec7e927fcf129"
server_key_version: "encryptionkey@0"
I20260808 16:52:27.933794 18627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/master-1/wal/instance:
uuid: "ca479471edff409d957dde1bb92c4144"
format_stamp: "Formatted at 2026-08-08 16:52:27 on dist-test-slave-6zbq"
server_key: "fc8a37e8e3c79aa195396dc82db853c2"
server_key_iv: "46ef5e14187bc3bc6f4ec7e927fcf129"
server_key_version: "encryptionkey@0"
I20260808 16:52:27.934978 18627 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 16:52:27.935618 18647 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:27.935760 18627 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 16:52:27.935835 18627 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/master-1/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/master-1/wal
uuid: "ca479471edff409d957dde1bb92c4144"
format_stamp: "Formatted at 2026-08-08 16:52:27 on dist-test-slave-6zbq"
server_key: "fc8a37e8e3c79aa195396dc82db853c2"
server_key_iv: "46ef5e14187bc3bc6f4ec7e927fcf129"
server_key_version: "encryptionkey@0"
I20260808 16:52:27.935891 18627 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:52:27.944324 18627 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:52:27.945784 18627 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:52:27.945899 18627 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:52:27.949491 18627 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.61:45755
I20260808 16:52:27.949522 18735 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.61:45755 every 8 connection(s)
I20260808 16:52:27.949800 18627 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/master-1/data/info.pb
I20260808 16:52:27.950672 18736 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:52:27.951953 18736 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.24.62" port: 46193 } has no permanent_uuid. Determining permanent_uuid...
I20260808 16:52:27.956396 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 18627
Aug 08 16:52:27 dist-test-slave-6zbq krb5kdc[17768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207947, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.24.61@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM
I20260808 16:52:27.960057 17917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:27.952523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58978 (local address 127.8.24.62:46193)
0808 16:52:27.952584 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:27.952586 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:27.952739 (+ 153us) server_negotiation.cc:407] Connection header received
0808 16:52:27.953195 (+ 456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:27.953196 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:27.953210 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:27.953261 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:27.953803 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:27.954290 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:27.955236 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:27.955418 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:27.955472 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:27.957582 (+ 2110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:27.957583 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:27.957584 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:27.957590 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:27.958836 (+ 1246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:27.959183 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:27.959185 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:27.959185 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:27.959237 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:27.959449 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:27.959450 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:27.959450 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:27.959588 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:27.959654 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:27.959887 (+ 233us) server_negotiation.cc:299] Negotiation successful
0808 16:52:27.959967 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 16:52:27.960180 18740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:27.952474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:46193 (local address 127.0.0.1:58978)
0808 16:52:27.952632 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:27.952670 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:27.953091 (+ 421us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:27.953327 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:27.953330 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:27.953543 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:27.953719 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:27.953722 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:27.954390 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:27.954391 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:27.955149 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:27.955151 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:27.955484 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:27.955485 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:27.955598 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:27.956026 (+ 428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:27.956034 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:27.957502 (+ 1468us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:52:27.958909 (+ 1407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:27.958915 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:27.958916 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:27.959112 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:27.959290 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:27.959291 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:27.959291 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:27.959393 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:27.959647 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:52:27.959652 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:52:27.959775 (+ 123us) client_negotiation.cc:769] Sending connection context
0808 16:52:27.959855 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 16:52:27.959984 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":77,"threads_started":1}
I20260808 16:52:27.960749 18736 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.24.61" port: 45755 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 16:52:27 dist-test-slave-6zbq krb5kdc[17768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207947, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.24.61@KRBTEST.COM
I20260808 16:52:27.964354 18742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:27.956831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55242 (local address 127.8.24.61:45755)
0808 16:52:27.956966 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:27.956970 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:27.957010 (+ 40us) server_negotiation.cc:407] Connection header received
0808 16:52:27.957074 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:27.957080 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:27.957102 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:27.957191 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:27.957882 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:27.958558 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:27.959249 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:27.959518 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:27.959574 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:27.961779 (+ 2205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:27.961786 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:27.961786 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:27.961792 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:27.963297 (+ 1505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:27.963675 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:27.963678 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:27.963679 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:27.963731 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:27.963882 (+ 151us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:27.963883 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:27.963883 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:27.964034 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:27.964153 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:27.964239 (+ 86us) server_negotiation.cc:299] Negotiation successful
0808 16:52:27.964294 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":63,"threads_started":1}
Aug 08 16:52:27 dist-test-slave-6zbq krb5kdc[17768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207947, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.24.61@KRBTEST.COM for oryx/127.8.24.61@KRBTEST.COM
I20260808 16:52:27.967432 18740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:27.960959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.61:45755 (local address 127.0.0.1:55248)
0808 16:52:27.961127 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:27.961137 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:27.961191 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:27.961310 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:27.961311 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:27.961444 (+ 133us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:27.961491 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:27.961493 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:27.962136 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:27.962137 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:27.962863 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:27.962866 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:27.963191 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:27.963192 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:27.963264 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:27.963589 (+ 325us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:27.963593 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:27.964821 (+ 1228us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:52:27.966628 (+ 1807us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:27.966629 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:27.966629 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:27.966773 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:27.966925 (+ 152us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:27.966926 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:27.966926 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:27.966978 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:27.967163 (+ 185us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:52:27.967164 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:52:27.967198 (+ 34us) client_negotiation.cc:769] Sending connection context
0808 16:52:27.967256 (+ 58us) client_negotiation.cc:240] Negotiation successful
0808 16:52:27.967341 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155}
I20260808 16:52:27.967501 18743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:27.961021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55248 (local address 127.8.24.61:45755)
0808 16:52:27.961142 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:27.961144 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:27.961178 (+ 34us) server_negotiation.cc:407] Connection header received
0808 16:52:27.961241 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:27.961241 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:27.961249 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:27.961301 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:27.961572 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:27.962062 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:27.962936 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:27.963124 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:27.963174 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:27.964889 (+ 1715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:27.964890 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:27.964891 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:27.964897 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:27.966562 (+ 1665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:27.966829 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:27.966830 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:27.966831 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:27.966877 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:27.967022 (+ 145us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:27.967023 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:27.967023 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:27.967111 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:27.967164 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:27.967280 (+ 116us) server_negotiation.cc:299] Negotiation successful
0808 16:52:27.967390 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":50,"server-negotiator.queue_time_us":105,"thread_start_us":66,"threads_started":1}
I20260808 16:52:27.968531 18736 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca479471edff409d957dde1bb92c4144: Bootstrap starting.
I20260808 16:52:27.969013 18736 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ca479471edff409d957dde1bb92c4144: Neither blocks nor log segments found. Creating new log.
I20260808 16:52:27.969259 18736 log.cc:824] T 00000000000000000000000000000000 P ca479471edff409d957dde1bb92c4144: Log is configured to *not* fsync() on all Append() calls
I20260808 16:52:27.969842 18736 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca479471edff409d957dde1bb92c4144: No bootstrap required, opened a new log
I20260808 16:52:27.971132 18736 raft_consensus.cc:348] T 00000000000000000000000000000000 P ca479471edff409d957dde1bb92c4144 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "678b35993e9b49aca0f3ad2bba5edfbd" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 46193 } } peers { permanent_uuid: "ca479471edff409d957dde1bb92c4144" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 45755 } }
I20260808 16:52:27.971233 18736 raft_consensus.cc:374] T 00000000000000000000000000000000 P ca479471edff409d957dde1bb92c4144 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:52:27.971253 18736 raft_consensus.cc:729] T 00000000000000000000000000000000 P ca479471edff409d957dde1bb92c4144 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca479471edff409d957dde1bb92c4144, State: Initialized, Role: FOLLOWER
I20260808 16:52:27.971342 18736 consensus_queue.cc:260] T 00000000000000000000000000000000 P ca479471edff409d957dde1bb92c4144 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "678b35993e9b49aca0f3ad2bba5edfbd" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 46193 } } peers { permanent_uuid: "ca479471edff409d957dde1bb92c4144" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 45755 } }
I20260808 16:52:27.971572 18736 sys_catalog.cc:564] T 00000000000000000000000000000000 P ca479471edff409d957dde1bb92c4144 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 16:52:27.971568 18745 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca479471edff409d957dde1bb92c4144 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "678b35993e9b49aca0f3ad2bba5edfbd" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 46193 } } peers { permanent_uuid: "ca479471edff409d957dde1bb92c4144" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 45755 } } }
I20260808 16:52:27.971653 18745 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca479471edff409d957dde1bb92c4144 [sys.catalog]: This master's current role is: FOLLOWER
W20260808 16:52:27.972960 18764 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ca479471edff409d957dde1bb92c4144: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 16:52:27.973013 18764 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 16:52:27.974607 18627 master_runner.cc:428] Detected that this master ca479471edff409d957dde1bb92c4144 is joining an existing cluster
I20260808 16:52:27.974686 18627 master_runner.cc:432] Initiating AddMaster RPC to add 127.8.24.61:45755
I20260808 16:52:27.980727 17917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:27.975551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58992 (local address 127.8.24.62:46193)
0808 16:52:27.975584 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:27.975586 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:27.975718 (+ 132us) server_negotiation.cc:407] Connection header received
0808 16:52:27.975814 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:27.975815 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:27.975825 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:27.975869 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:27.976183 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:27.976619 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:27.977349 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:27.977539 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:27.977589 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:27.978330 (+ 741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:27.978331 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:27.978332 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:27.978355 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:27.979579 (+ 1224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:27.979913 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:27.979914 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:27.979915 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:27.979967 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:27.980239 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:27.980239 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:27.980240 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:27.980334 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:27.980384 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:27.980560 (+ 176us) server_negotiation.cc:299] Negotiation successful
0808 16:52:27.980638 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260808 16:52:27.980738 18774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:27.975518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:46193 (local address 127.0.0.1:58992)
0808 16:52:27.975658 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:27.975667 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:27.975741 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:27.975883 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:27.975883 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:27.976055 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:27.976118 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:27.976119 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:27.976683 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:27.976684 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:27.977280 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:27.977282 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:27.977627 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:27.977628 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:27.977686 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:27.978000 (+ 314us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:27.978005 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:27.978255 (+ 250us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:52:27.979666 (+ 1411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:27.979667 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:27.979668 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:27.979847 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:27.980021 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:27.980022 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:27.980022 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:27.980078 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:27.980392 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:52:27.980393 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:52:27.980432 (+ 39us) client_negotiation.cc:769] Sending connection context
0808 16:52:27.980524 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 16:52:27.980645 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":87,"threads_started":1}
I20260808 16:52:27.982594 18743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:27.975695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55262 (local address 127.8.24.61:45755)
0808 16:52:27.975721 (+ 26us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:27.975722 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:27.975762 (+ 40us) server_negotiation.cc:407] Connection header received
0808 16:52:27.975905 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:27.975906 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:27.975913 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:27.975960 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:27.976313 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:27.976994 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:27.977977 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:27.978266 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:27.978304 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:27.979331 (+ 1027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:27.979332 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:27.979332 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:27.979338 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:27.980841 (+ 1503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:27.981795 (+ 954us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:27.981795 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:27.981796 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:27.981852 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:27.982072 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:27.982073 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:27.982073 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:27.982181 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:27.982218 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:27.982440 (+ 222us) server_negotiation.cc:299] Negotiation successful
0808 16:52:27.982512 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13}
I20260808 16:52:27.982868 18778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:27.975521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.61:45755 (local address 127.0.0.1:55262)
0808 16:52:27.975708 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:27.975717 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:27.975774 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:27.975975 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:27.975976 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:27.976169 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:27.976229 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:27.976231 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:27.977086 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:27.977087 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:27.977890 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:27.977892 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:27.978395 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:27.978396 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:27.978464 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:27.978903 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:27.978908 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:27.979242 (+ 334us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:52:27.981295 (+ 2053us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:27.981296 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:27.981297 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:27.981495 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:27.981923 (+ 428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:27.981924 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:27.981925 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:27.981997 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:27.982255 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:52:27.982256 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:52:27.982299 (+ 43us) client_negotiation.cc:769] Sending connection context
0808 16:52:27.982703 (+ 404us) client_negotiation.cc:240] Negotiation successful
0808 16:52:27.982783 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":91,"threads_started":1}
I20260808 16:52:27.985491 18745 raft_consensus.cc:482] T 00000000000000000000000000000000 P ca479471edff409d957dde1bb92c4144 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 16:52:27.985569 18745 raft_consensus.cc:504] T 00000000000000000000000000000000 P ca479471edff409d957dde1bb92c4144 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "678b35993e9b49aca0f3ad2bba5edfbd" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 46193 } } peers { permanent_uuid: "ca479471edff409d957dde1bb92c4144" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 45755 } }
I20260808 16:52:27.985752 18745 leader_election.cc:288] T 00000000000000000000000000000000 P ca479471edff409d957dde1bb92c4144 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 678b35993e9b49aca0f3ad2bba5edfbd (127.8.24.62:46193)
I20260808 16:52:27.985944 17858 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ca479471edff409d957dde1bb92c4144" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "678b35993e9b49aca0f3ad2bba5edfbd" is_pre_election: true
I20260808 16:52:27.986245 18649 leader_election.cc:302] T 00000000000000000000000000000000 P ca479471edff409d957dde1bb92c4144 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: ca479471edff409d957dde1bb92c4144; no voters: 678b35993e9b49aca0f3ad2bba5edfbd
I20260808 16:52:27.986390 18745 raft_consensus.cc:2726] T 00000000000000000000000000000000 P ca479471edff409d957dde1bb92c4144 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
Aug 08 16:52:27 dist-test-slave-6zbq krb5kdc[17768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207947, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.24.62@KRBTEST.COM for oryx/127.8.24.61@KRBTEST.COM
I20260808 16:52:27.990163 18743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:27.983210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55266 (local address 127.8.24.61:45755)
0808 16:52:27.983312 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:27.983314 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:27.983320 (+ 6us) server_negotiation.cc:407] Connection header received
0808 16:52:27.983340 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:27.983341 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:27.983349 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:27.983407 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:27.983737 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:27.984423 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:27.985083 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:27.985367 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:27.985429 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:27.988022 (+ 2593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:27.988023 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:27.988023 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:27.988029 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:27.989163 (+ 1134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:27.989565 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:27.989565 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:27.989565 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:27.989603 (+ 38us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:27.989769 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:27.989770 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:27.989770 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:27.989845 (+ 75us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:27.989887 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:27.990056 (+ 169us) server_negotiation.cc:299] Negotiation successful
0808 16:52:27.990103 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260808 16:52:27.990236 18779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:27.983056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.61:45755 (local address 127.0.0.1:55266)
0808 16:52:27.983178 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:27.983217 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:27.983271 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:27.983410 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:27.983412 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:27.983622 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:27.983671 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:27.983673 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:27.984484 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:27.984485 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:27.985015 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:27.985016 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:27.985835 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:27.985836 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:27.986177 (+ 341us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:27.986596 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:27.986601 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:27.987944 (+ 1343us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:52:27.989231 (+ 1287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:27.989237 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:27.989239 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:27.989502 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:27.989652 (+ 150us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:27.989652 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:27.989653 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:27.989721 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:27.989895 (+ 174us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:52:27.989900 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:52:27.989951 (+ 51us) client_negotiation.cc:769] Sending connection context
0808 16:52:27.990032 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 16:52:27.990142 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":61,"threads_started":1}
I20260808 16:52:27.990891 17840 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.8.24.61:45755: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "ca479471edff409d957dde1bb92c4144"
member_type: NON_VOTER
last_known_addr {
host: "127.8.24.61"
port: 45755
}
attrs {
promote: true
}
}
dest_uuid: "678b35993e9b49aca0f3ad2bba5edfbd"
cas_config_opid_index: -1
I20260808 16:52:27.991163 17840 consensus_queue.cc:237] T 00000000000000000000000000000000 P 678b35993e9b49aca0f3ad2bba5edfbd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "678b35993e9b49aca0f3ad2bba5edfbd" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 46193 } } peers { permanent_uuid: "ca479471edff409d957dde1bb92c4144" member_type: NON_VOTER last_known_addr { host: "127.8.24.61" port: 45755 } attrs { promote: true } }
I20260808 16:52:27.991611 18782 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 678b35993e9b49aca0f3ad2bba5edfbd [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER ca479471edff409d957dde1bb92c4144 (127.8.24.61) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "678b35993e9b49aca0f3ad2bba5edfbd" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 46193 } } peers { permanent_uuid: "ca479471edff409d957dde1bb92c4144" member_type: NON_VOTER last_known_addr { host: "127.8.24.61" port: 45755 } attrs { promote: true } } }
I20260808 16:52:27.991791 18692 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ca479471edff409d957dde1bb92c4144 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:52:27.992395 18692 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ca479471edff409d957dde1bb92c4144 [term 1 FOLLOWER]: Refusing update from remote peer 678b35993e9b49aca0f3ad2bba5edfbd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260808 16:52:27.992545 18782 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.8.24.61:45755
I20260808 16:52:27.992805 18627 master.cc:537] Master@127.8.24.61:45755 shutting down...
I20260808 16:52:27.992705 18784 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 678b35993e9b49aca0f3ad2bba5edfbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca479471edff409d957dde1bb92c4144" member_type: NON_VOTER last_known_addr { host: "127.8.24.61" port: 45755 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:52:27.992741 18780 sys_catalog.cc:455] T 00000000000000000000000000000000 P 678b35993e9b49aca0f3ad2bba5edfbd [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "678b35993e9b49aca0f3ad2bba5edfbd" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "678b35993e9b49aca0f3ad2bba5edfbd" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 46193 } } peers { permanent_uuid: "ca479471edff409d957dde1bb92c4144" member_type: NON_VOTER last_known_addr { host: "127.8.24.61" port: 45755 } attrs { promote: true } } }
I20260808 16:52:27.993589 18780 sys_catalog.cc:458] T 00000000000000000000000000000000 P 678b35993e9b49aca0f3ad2bba5edfbd [sys.catalog]: This master's current role is: LEADER
W20260808 16:52:27.993848 17821 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260808 16:52:27.994249 17821 consensus_peers.cc:487] T 00000000000000000000000000000000 P 678b35993e9b49aca0f3ad2bba5edfbd -> Peer ca479471edff409d957dde1bb92c4144 (127.8.24.61:45755): Couldn't send request to peer ca479471edff409d957dde1bb92c4144. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260808 16:52:27.995242 18627 raft_consensus.cc:2229] T 00000000000000000000000000000000 P ca479471edff409d957dde1bb92c4144 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 16:52:27.995302 18627 raft_consensus.cc:2258] T 00000000000000000000000000000000 P ca479471edff409d957dde1bb92c4144 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:52:27.995343 18627 tablet_replica.cc:330] T 00000000000000000000000000000000 P ca479471edff409d957dde1bb92c4144: stopping tablet replica
I20260808 16:52:28.007710 18627 master.cc:552] Master@127.8.24.61:45755 shutdown complete.
I20260808 16:52:28.007745 18627 master_runner.cc:305] Clearing existing system tablet
I20260808 16:52:28.007853 18627 fs_manager.cc:359] Metadata directory not provided
I20260808 16:52:28.007887 18627 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:52:28.008352 18627 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260808 16:52:28.008749 18791 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:28.008864 18627 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:52:28.008932 18627 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:28.008957 18627 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/master-1/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/master-1/wal
uuid: "ca479471edff409d957dde1bb92c4144"
format_stamp: "Formatted at 2026-08-08 16:52:27 on dist-test-slave-6zbq"
server_key: "fc8a37e8e3c79aa195396dc82db853c2"
server_key_iv: "46ef5e14187bc3bc6f4ec7e927fcf129"
server_key_version: "encryptionkey@0"
I20260808 16:52:28.009744 18627 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P ca479471edff409d957dde1bb92c4144: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:52:28.011020 18627 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P ca479471edff409d957dde1bb92c4144: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260808 16:52:28.011088 18627 log.cc:1196] T 00000000000000000000000000000000 P ca479471edff409d957dde1bb92c4144: Deleting WAL directory at /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260808 16:52:28.011273 18627 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P ca479471edff409d957dde1bb92c4144: Deleting consensus metadata
I20260808 16:52:28.011497 18627 master_runner.cc:315] Copying system tablet from 127.8.24.62:46193
I20260808 16:52:28.012039 18627 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P ca479471edff409d957dde1bb92c4144: tablet copy: Beginning tablet copy session from remote peer at address 127.8.24.62:46193
I20260808 16:52:28.017431 18801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:28.012306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:46193 (local address 127.0.0.1:59002)
0808 16:52:28.012398 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:28.012407 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:28.012450 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:28.012562 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:28.012563 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:28.012698 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:28.012743 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:28.012744 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:28.013569 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:28.013569 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:28.014072 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:28.014073 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:28.014504 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:28.014505 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:28.014548 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:28.014885 (+ 337us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:28.014889 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:28.015152 (+ 263us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:52:28.016664 (+ 1512us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:28.016664 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:28.016665 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:28.016808 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:28.016989 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:28.016989 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:28.016989 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:28.017030 (+ 41us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:28.017235 (+ 205us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:52:28.017235 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:52:28.017264 (+ 29us) client_negotiation.cc:769] Sending connection context
0808 16:52:28.017304 (+ 40us) client_negotiation.cc:240] Negotiation successful
0808 16:52:28.017365 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":74,"thread_start_us":59,"threads_started":1}
I20260808 16:52:28.017478 17917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:28.012365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59002 (local address 127.8.24.62:46193)
0808 16:52:28.012407 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:28.012408 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:28.012444 (+ 36us) server_negotiation.cc:407] Connection header received
0808 16:52:28.012486 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:28.012487 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:28.012495 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:28.012559 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:28.012796 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:28.013503 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:28.014131 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:28.014444 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:28.014506 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:28.015273 (+ 767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:28.015275 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:28.015275 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:28.015282 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:28.016579 (+ 1297us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:28.016866 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:28.016867 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:28.016867 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:28.016933 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:28.017066 (+ 133us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:28.017066 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:28.017066 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:28.017179 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:28.017246 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:28.017328 (+ 82us) server_negotiation.cc:299] Negotiation successful
0808 16:52:28.017396 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 16:52:28.017730 17878 tablet_copy_service.cc:140] P 678b35993e9b49aca0f3ad2bba5edfbd: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer ca479471edff409d957dde1bb92c4144 ({username='oryx', principal='oryx/127.8.24.61@KRBTEST.COM'} at 127.0.0.1:59002)
I20260808 16:52:28.017840 17878 tablet_copy_service.cc:161] P 678b35993e9b49aca0f3ad2bba5edfbd: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer ca479471edff409d957dde1bb92c4144 at {username='oryx', principal='oryx/127.8.24.61@KRBTEST.COM'} at 127.0.0.1:59002: session id = ca479471edff409d957dde1bb92c4144-00000000000000000000000000000000
I20260808 16:52:28.018679 17878 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 678b35993e9b49aca0f3ad2bba5edfbd: Tablet Copy: opened 0 blocks and 1 log segments
I20260808 16:52:28.019215 18627 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260808 16:52:28.020287 18627 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P ca479471edff409d957dde1bb92c4144: tablet copy: Starting download of 0 data blocks...
I20260808 16:52:28.020421 18627 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P ca479471edff409d957dde1bb92c4144: tablet copy: Starting download of 1 WAL segments...
I20260808 16:52:28.021240 18627 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P ca479471edff409d957dde1bb92c4144: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260808 16:52:28.022027 17878 tablet_copy_service.cc:342] P 678b35993e9b49aca0f3ad2bba5edfbd: Request end of tablet copy session ca479471edff409d957dde1bb92c4144-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.8.24.61@KRBTEST.COM'} at 127.0.0.1:59002
I20260808 16:52:28.022080 17878 tablet_copy_service.cc:433] P 678b35993e9b49aca0f3ad2bba5edfbd: ending tablet copy session ca479471edff409d957dde1bb92c4144-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer ca479471edff409d957dde1bb92c4144
I20260808 16:52:28.024153 18627 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:52:28.025044 18803 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:28.025120 18806 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:28.025309 18804 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:28.026476 18627 server_base.cc:1034] running on GCE node
I20260808 16:52:28.026557 18627 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:52:28.026676 18627 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:52:28.027788 18627 hybrid_clock.cc:648] HybridClock initialized: now 1786207948027781 us; error 22 us; skew 500 ppm
Aug 08 16:52:28 dist-test-slave-6zbq krb5kdc[17768](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207948, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.24.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:52:28.028821 18627 init.cc:377] Logged in from keytab as oryx/127.8.24.61@KRBTEST.COM (short username oryx)
I20260808 16:52:28.029039 18627 webserver.cc:460] Webserver started at http://127.8.24.61:40091/ using document root <none> and password file <none>
I20260808 16:52:28.029143 18627 fs_manager.cc:359] Metadata directory not provided
I20260808 16:52:28.029177 18627 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:52:28.029664 18627 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260808 16:52:28.030114 18821 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:28.030230 18627 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:28.030270 18627 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/master-1/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/master-1/wal
uuid: "ca479471edff409d957dde1bb92c4144"
format_stamp: "Formatted at 2026-08-08 16:52:27 on dist-test-slave-6zbq"
server_key: "fc8a37e8e3c79aa195396dc82db853c2"
server_key_iv: "46ef5e14187bc3bc6f4ec7e927fcf129"
server_key_version: "encryptionkey@0"
I20260808 16:52:28.030310 18627 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:52:28.039942 18627 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:52:28.041309 18627 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:52:28.044011 18627 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.61:45755
I20260808 16:52:28.044179 18902 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.61:45755 every 8 connection(s)
I20260808 16:52:28.044294 18627 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/master-1/data/info.pb
I20260808 16:52:28.045084 18903 sys_catalog.cc:263] Verifying existing consensus state
I20260808 16:52:28.045565 18903 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca479471edff409d957dde1bb92c4144: Bootstrap starting.
I20260808 16:52:28.048439 18903 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca479471edff409d957dde1bb92c4144: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 16:52:28.048697 18903 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca479471edff409d957dde1bb92c4144: Bootstrap complete.
I20260808 16:52:28.048882 18903 raft_consensus.cc:348] T 00000000000000000000000000000000 P ca479471edff409d957dde1bb92c4144 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "678b35993e9b49aca0f3ad2bba5edfbd" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 46193 } } peers { permanent_uuid: "ca479471edff409d957dde1bb92c4144" member_type: NON_VOTER last_known_addr { host: "127.8.24.61" port: 45755 } attrs { promote: true } }
I20260808 16:52:28.048959 18903 raft_consensus.cc:729] T 00000000000000000000000000000000 P ca479471edff409d957dde1bb92c4144 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ca479471edff409d957dde1bb92c4144, State: Initialized, Role: LEARNER
I20260808 16:52:28.049003 18903 consensus_queue.cc:260] T 00000000000000000000000000000000 P ca479471edff409d957dde1bb92c4144 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "678b35993e9b49aca0f3ad2bba5edfbd" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 46193 } } peers { permanent_uuid: "ca479471edff409d957dde1bb92c4144" member_type: NON_VOTER last_known_addr { host: "127.8.24.61" port: 45755 } attrs { promote: true } }
I20260808 16:52:28.049171 18903 sys_catalog.cc:564] T 00000000000000000000000000000000 P ca479471edff409d957dde1bb92c4144 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 16:52:28.049161 18909 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca479471edff409d957dde1bb92c4144 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "678b35993e9b49aca0f3ad2bba5edfbd" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 46193 } } peers { permanent_uuid: "ca479471edff409d957dde1bb92c4144" member_type: NON_VOTER last_known_addr { host: "127.8.24.61" port: 45755 } attrs { promote: true } } }
I20260808 16:52:28.049250 18909 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca479471edff409d957dde1bb92c4144 [sys.catalog]: This master's current role is: LEARNER
I20260808 16:52:28.050660 18922 catalog_manager.cc:1259] Loaded cluster ID: ddde3755643b4afbb76aa4f0f8896bc8
I20260808 16:52:28.050700 18922 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ca479471edff409d957dde1bb92c4144: loading cluster ID for follower catalog manager: success
I20260808 16:52:28.051537 18922 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ca479471edff409d957dde1bb92c4144: acquiring CA information for follower catalog manager: success
I20260808 16:52:28.051776 18922 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ca479471edff409d957dde1bb92c4144: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 16:52:28.071388 18924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:28.065671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55284 (local address 127.8.24.61:45755)
0808 16:52:28.065770 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:28.065771 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:28.065778 (+ 7us) server_negotiation.cc:407] Connection header received
0808 16:52:28.065794 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:28.065795 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:28.065802 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:28.065858 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:28.066355 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:28.066981 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:28.067724 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:28.067917 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:28.067969 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:28.069041 (+ 1072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:28.069043 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:28.069043 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:28.069050 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:28.070243 (+ 1193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:28.070590 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:28.070591 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:28.070592 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:28.070649 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:28.070808 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:28.070809 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:28.070809 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:28.070901 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:28.070980 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:28.071085 (+ 105us) server_negotiation.cc:299] Negotiation successful
0808 16:52:28.071156 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79,"thread_start_us":51,"threads_started":1}
WARNING: no policy specified for oryx/127.8.24.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.24.60@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.24.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/krb5kdc/oryx_127.8.24.60.keytab.
Entry for principal oryx/127.8.24.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/krb5kdc/oryx_127.8.24.60.keytab.
I20260808 16:52:28.094533 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.60:39771
--webserver_interface=127.8.24.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/krb5kdc/oryx_127.8.24.60.keytab
--principal=oryx/127.8.24.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:37767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:46193,127.8.24.61:45755,127.8.24.60:39771
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:52:28.172540 18928 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:52:28.172717 18928 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:52:28.172739 18928 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:52:28.173770 18928 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 16:52:28.173802 18928 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:52:28.173816 18928 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:52:28.173827 18928 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 16:52:28.173840 18928 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 16:52:28.175261 18928 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:37767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:46193,127.8.24.61:45755,127.8.24.60:39771
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/krb5kdc/oryx_127.8.24.60.keytab
--principal=oryx/127.8.24.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.60:39771
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.8.24.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18928
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:52:28.175431 18928 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:52:28.175585 18928 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:52:28.179849 18937 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:28.179836 18934 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:28.179836 18942 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:28.179867 18928 server_base.cc:1034] running on GCE node
I20260808 16:52:28.180267 18928 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:52:28.180523 18928 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:52:28.181654 18928 hybrid_clock.cc:648] HybridClock initialized: now 1786207948181641 us; error 28 us; skew 500 ppm
Aug 08 16:52:28 dist-test-slave-6zbq krb5kdc[17768](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207948, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.24.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:52:28.183858 18928 init.cc:377] Logged in from keytab as oryx/127.8.24.60@KRBTEST.COM (short username oryx)
I20260808 16:52:28.184260 18928 webserver.cc:460] Webserver started at http://127.8.24.60:45729/ using document root <none> and password file <none>
I20260808 16:52:28.184438 18928 fs_manager.cc:359] Metadata directory not provided
I20260808 16:52:28.184480 18928 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:52:28.184599 18928 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:52:28.185527 18928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/master-2/data/instance:
uuid: "2a210ec3273f40d1b29270637e35af7b"
format_stamp: "Formatted at 2026-08-08 16:52:28 on dist-test-slave-6zbq"
server_key: "27363af375b22d2d447906e0d11fabe2"
server_key_iv: "51342327929f33f9b3867ff39a965cb5"
server_key_version: "encryptionkey@0"
I20260808 16:52:28.185840 18928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/master-2/wal/instance:
uuid: "2a210ec3273f40d1b29270637e35af7b"
format_stamp: "Formatted at 2026-08-08 16:52:28 on dist-test-slave-6zbq"
server_key: "27363af375b22d2d447906e0d11fabe2"
server_key_iv: "51342327929f33f9b3867ff39a965cb5"
server_key_version: "encryptionkey@0"
I20260808 16:52:28.187071 18928 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 16:52:28.187690 18956 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:28.187839 18928 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 16:52:28.187920 18928 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/master-2/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/master-2/wal
uuid: "2a210ec3273f40d1b29270637e35af7b"
format_stamp: "Formatted at 2026-08-08 16:52:28 on dist-test-slave-6zbq"
server_key: "27363af375b22d2d447906e0d11fabe2"
server_key_iv: "51342327929f33f9b3867ff39a965cb5"
server_key_version: "encryptionkey@0"
I20260808 16:52:28.187971 18928 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:52:28.206099 18928 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:52:28.207868 18928 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:52:28.207965 18928 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:52:28.211468 18928 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.60:39771
I20260808 16:52:28.211503 19052 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.60:39771 every 8 connection(s)
I20260808 16:52:28.211783 18928 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/master-2/data/info.pb
I20260808 16:52:28.212646 19053 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:52:28.213948 19053 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.24.62" port: 46193 } has no permanent_uuid. Determining permanent_uuid...
I20260808 16:52:28.218319 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 18928
Aug 08 16:52:28 dist-test-slave-6zbq krb5kdc[17768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207948, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.24.60@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM
I20260808 16:52:28.221757 17917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:28.214631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59004 (local address 127.8.24.62:46193)
0808 16:52:28.214701 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:28.214703 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:28.214778 (+ 75us) server_negotiation.cc:407] Connection header received
0808 16:52:28.215180 (+ 402us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:28.215181 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:28.215190 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:28.215238 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:28.215744 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:28.216220 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:28.217103 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:28.217276 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:28.217316 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:28.219363 (+ 2047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:28.219364 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:28.219365 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:28.219373 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:28.220557 (+ 1184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:28.220908 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:28.220910 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:28.220910 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:28.220961 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:28.221190 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:28.221191 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:28.221191 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:28.221292 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:28.221347 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:28.221590 (+ 243us) server_negotiation.cc:299] Negotiation successful
0808 16:52:28.221670 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 16:52:28.221885 19058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:28.214502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:46193 (local address 127.0.0.1:59004)
0808 16:52:28.214673 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:28.214713 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:28.215097 (+ 384us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:28.215295 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:28.215298 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:28.215510 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:28.215684 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:28.215688 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:28.216275 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:28.216276 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:28.217041 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:28.217043 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:28.217356 (+ 313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:28.217357 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:28.217465 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:28.217825 (+ 360us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:28.217833 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:28.219282 (+ 1449us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:52:28.220632 (+ 1350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:28.220638 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:28.220640 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:28.220842 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:28.221016 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:28.221017 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:28.221017 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:28.221130 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:28.221348 (+ 218us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:52:28.221354 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:52:28.221475 (+ 121us) client_negotiation.cc:769] Sending connection context
0808 16:52:28.221557 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 16:52:28.221697 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":70,"threads_started":1}
I20260808 16:52:28.222374 19053 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.24.61" port: 45755 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 16:52:28 dist-test-slave-6zbq krb5kdc[17768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207947, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.24.60@KRBTEST.COM
Aug 08 16:52:28 dist-test-slave-6zbq krb5kdc[17768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207948, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.24.60@KRBTEST.COM for oryx/127.8.24.61@KRBTEST.COM
I20260808 16:52:28.226227 19059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:28.218880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55744 (local address 127.8.24.60:39771)
0808 16:52:28.219039 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:28.219043 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:28.219058 (+ 15us) server_negotiation.cc:407] Connection header received
0808 16:52:28.219075 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:28.219077 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:28.219092 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:28.219151 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:28.219683 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:28.220311 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:28.220954 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:28.221246 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:28.221301 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:28.223817 (+ 2516us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:28.223824 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:28.223824 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:28.223830 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:28.225225 (+ 1395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:28.225518 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:28.225521 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:28.225521 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:28.225573 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:28.225737 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:28.225738 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:28.225739 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:28.225863 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:28.226008 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:28.226059 (+ 51us) server_negotiation.cc:299] Negotiation successful
0808 16:52:28.226122 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":74,"threads_started":1}
I20260808 16:52:28.227995 19058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:28.222587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.61:45755 (local address 127.0.0.1:55294)
0808 16:52:28.222626 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:28.222634 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:28.222664 (+ 30us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:28.222831 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:28.222832 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:28.222961 (+ 129us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:28.223010 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:28.223012 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:28.223739 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:28.223740 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:28.224216 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:28.224217 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:28.224551 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:28.224552 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:28.224619 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:28.224922 (+ 303us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:28.224925 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:28.225912 (+ 987us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:52:28.227317 (+ 1405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:28.227317 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:28.227318 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:28.227465 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:28.227607 (+ 142us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:28.227607 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:28.227608 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:28.227656 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:28.227811 (+ 155us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:52:28.227812 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:52:28.227841 (+ 29us) client_negotiation.cc:769] Sending connection context
0808 16:52:28.227886 (+ 45us) client_negotiation.cc:240] Negotiation successful
0808 16:52:28.227944 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260808 16:52:28.228014 18924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:28.222678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55294 (local address 127.8.24.61:45755)
0808 16:52:28.222734 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:28.222736 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:28.222742 (+ 6us) server_negotiation.cc:407] Connection header received
0808 16:52:28.222767 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:28.222768 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:28.222777 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:28.222821 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:28.223095 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:28.223649 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:28.224294 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:28.224495 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:28.224547 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:28.225986 (+ 1439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:28.225987 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:28.225988 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:28.225994 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:28.227242 (+ 1248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:28.227522 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:28.227522 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:28.227523 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:28.227568 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:28.227688 (+ 120us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:28.227689 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:28.227689 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:28.227770 (+ 81us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:28.227811 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:28.227902 (+ 91us) server_negotiation.cc:299] Negotiation successful
0808 16:52:28.227958 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 16:52:28.228334 19053 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.24.60" port: 39771 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 16:52:28 dist-test-slave-6zbq krb5kdc[17768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207948, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.24.60@KRBTEST.COM for oryx/127.8.24.60@KRBTEST.COM
I20260808 16:52:28.233363 19058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:28.228530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.60:39771 (local address 127.0.0.1:55754)
0808 16:52:28.228568 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:28.228577 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:28.228630 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:28.228720 (+ 90us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:28.228720 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:28.228832 (+ 112us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:28.228854 (+ 22us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:28.228855 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:28.229459 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:28.229460 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:28.229902 (+ 442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:28.229903 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:28.230284 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:28.230285 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:28.230331 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:28.230677 (+ 346us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:28.230682 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:28.231645 (+ 963us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:52:28.232739 (+ 1094us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:28.232740 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:28.232740 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:28.232880 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:28.233005 (+ 125us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:28.233005 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:28.233005 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:28.233044 (+ 39us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:28.233184 (+ 140us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:52:28.233184 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:52:28.233213 (+ 29us) client_negotiation.cc:769] Sending connection context
0808 16:52:28.233254 (+ 41us) client_negotiation.cc:240] Negotiation successful
0808 16:52:28.233313 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260808 16:52:28.233436 19059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:28.228555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55754 (local address 127.8.24.60:39771)
0808 16:52:28.228580 (+ 25us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:28.228582 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:28.228609 (+ 27us) server_negotiation.cc:407] Connection header received
0808 16:52:28.228669 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:28.228669 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:28.228677 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:28.228719 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:28.228909 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:28.229391 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:28.229963 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:28.230228 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:28.230279 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:28.231714 (+ 1435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:28.231716 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:28.231716 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:28.231721 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:28.232679 (+ 958us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:28.232931 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:28.232931 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:28.232932 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:28.232971 (+ 39us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:28.233074 (+ 103us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:28.233075 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:28.233075 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:28.233148 (+ 73us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:28.233189 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:28.233270 (+ 81us) server_negotiation.cc:299] Negotiation successful
0808 16:52:28.233353 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":32,"server-negotiator.queue_time_us":12}
I20260808 16:52:28.234411 19053 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a210ec3273f40d1b29270637e35af7b: Bootstrap starting.
I20260808 16:52:28.235122 19053 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2a210ec3273f40d1b29270637e35af7b: Neither blocks nor log segments found. Creating new log.
I20260808 16:52:28.235383 19053 log.cc:824] T 00000000000000000000000000000000 P 2a210ec3273f40d1b29270637e35af7b: Log is configured to *not* fsync() on all Append() calls
I20260808 16:52:28.235973 19053 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a210ec3273f40d1b29270637e35af7b: No bootstrap required, opened a new log
I20260808 16:52:28.237237 19053 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2a210ec3273f40d1b29270637e35af7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "678b35993e9b49aca0f3ad2bba5edfbd" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 46193 } } peers { permanent_uuid: "ca479471edff409d957dde1bb92c4144" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 45755 } } peers { permanent_uuid: "2a210ec3273f40d1b29270637e35af7b" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 39771 } }
I20260808 16:52:28.237350 19053 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2a210ec3273f40d1b29270637e35af7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:52:28.237372 19053 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2a210ec3273f40d1b29270637e35af7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a210ec3273f40d1b29270637e35af7b, State: Initialized, Role: FOLLOWER
I20260808 16:52:28.237485 19053 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2a210ec3273f40d1b29270637e35af7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "678b35993e9b49aca0f3ad2bba5edfbd" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 46193 } } peers { permanent_uuid: "ca479471edff409d957dde1bb92c4144" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 45755 } } peers { permanent_uuid: "2a210ec3273f40d1b29270637e35af7b" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 39771 } }
I20260808 16:52:28.237722 19053 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2a210ec3273f40d1b29270637e35af7b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 16:52:28.237715 19061 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a210ec3273f40d1b29270637e35af7b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "678b35993e9b49aca0f3ad2bba5edfbd" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 46193 } } peers { permanent_uuid: "ca479471edff409d957dde1bb92c4144" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 45755 } } peers { permanent_uuid: "2a210ec3273f40d1b29270637e35af7b" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 39771 } } }
I20260808 16:52:28.237789 19061 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a210ec3273f40d1b29270637e35af7b [sys.catalog]: This master's current role is: FOLLOWER
W20260808 16:52:28.239197 19078 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2a210ec3273f40d1b29270637e35af7b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 16:52:28.239243 19078 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 16:52:28.241034 18928 master_runner.cc:428] Detected that this master 2a210ec3273f40d1b29270637e35af7b is joining an existing cluster
I20260808 16:52:28.241082 18928 master_runner.cc:432] Initiating AddMaster RPC to add 127.8.24.60:39771
I20260808 16:52:28.248266 19091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:28.242259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.60:39771 (local address 127.0.0.1:55758)
0808 16:52:28.242622 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:28.242631 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:28.242692 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:28.242793 (+ 101us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:28.242793 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:28.242962 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:28.243007 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:28.243009 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:28.243819 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:28.243820 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:28.244708 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:28.244710 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:28.245157 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:28.245158 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:28.245209 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:28.245537 (+ 328us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:28.245541 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:28.245831 (+ 290us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:52:28.247451 (+ 1620us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:28.247452 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:28.247452 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:28.247595 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:28.247789 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:28.247790 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:28.247790 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:28.247831 (+ 41us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:28.248047 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:52:28.248048 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:52:28.248090 (+ 42us) client_negotiation.cc:769] Sending connection context
0808 16:52:28.248137 (+ 47us) client_negotiation.cc:240] Negotiation successful
0808 16:52:28.248204 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":338,"mutex_wait_us":47,"thread_start_us":64,"threads_started":1}
I20260808 16:52:28.248363 19059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:28.242435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55758 (local address 127.8.24.60:39771)
0808 16:52:28.242469 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:28.242471 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:28.242673 (+ 202us) server_negotiation.cc:407] Connection header received
0808 16:52:28.242734 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:28.242735 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:28.242742 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:28.242780 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:28.243071 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:28.243739 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:28.244792 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:28.245053 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:28.245087 (+ 34us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:28.245906 (+ 819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:28.245908 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:28.245908 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:28.245914 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:28.247378 (+ 1464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:28.247672 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:28.247673 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:28.247673 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:28.247731 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:28.247880 (+ 149us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:28.247880 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:28.247881 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:28.247994 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:28.248060 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:28.248161 (+ 101us) server_negotiation.cc:299] Negotiation successful
0808 16:52:28.248266 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":29,"server-negotiator.queue_time_us":18}
I20260808 16:52:28.265496 17917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:28.242071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59018 (local address 127.8.24.62:46193)
0808 16:52:28.242109 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:28.242111 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:28.242253 (+ 142us) server_negotiation.cc:407] Connection header received
0808 16:52:28.242365 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:28.242366 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:28.242375 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:28.242403 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:28.242895 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:28.243616 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:28.244904 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:28.245189 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:28.245228 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:28.263252 (+ 18024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:28.263253 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:28.263254 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:28.263261 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:28.264463 (+ 1202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:28.264747 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:28.264748 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:28.264748 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:28.264797 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:28.264983 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:28.264984 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:28.264984 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:28.265115 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:28.265177 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:28.265345 (+ 168us) server_negotiation.cc:299] Negotiation successful
0808 16:52:28.265417 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 16:52:28.265501 19089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:28.242034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:46193 (local address 127.0.0.1:59018)
0808 16:52:28.242185 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:28.242195 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:28.242276 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:28.242495 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:28.242496 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:28.242734 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:28.242805 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:28.242807 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:28.243821 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:28.243822 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:28.244804 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:28.244807 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:28.245297 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:28.245298 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:28.245356 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:28.262733 (+ 17377us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:28.262738 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:28.263164 (+ 426us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:52:28.264534 (+ 1370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:28.264535 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:28.264535 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:28.264687 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:28.264863 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:28.264864 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:28.264864 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:28.264914 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:28.265207 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:52:28.265208 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:52:28.265247 (+ 39us) client_negotiation.cc:769] Sending connection context
0808 16:52:28.265313 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 16:52:28.265398 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"mutex_wait_us":33,"thread_start_us":99,"threads_started":1}
I20260808 16:52:28.265782 18924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:28.242387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55304 (local address 127.8.24.61:45755)
0808 16:52:28.242474 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:28.242476 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:28.242484 (+ 8us) server_negotiation.cc:407] Connection header received
0808 16:52:28.242501 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:28.242502 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:28.242509 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:28.242552 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:28.243039 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:28.243739 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:28.244821 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:28.245104 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:28.245147 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:28.263285 (+ 18138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:28.263286 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:28.263287 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:28.263293 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:28.264760 (+ 1467us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:28.265066 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:28.265067 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:28.265067 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:28.265118 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:28.265273 (+ 155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:28.265274 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:28.265274 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:28.265383 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:28.265439 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:28.265624 (+ 185us) server_negotiation.cc:299] Negotiation successful
0808 16:52:28.265705 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260808 16:52:28.266059 19093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:28.242040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.61:45755 (local address 127.0.0.1:55304)
0808 16:52:28.242399 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:28.242409 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:28.242456 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:28.242555 (+ 99us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:28.242556 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:28.242885 (+ 329us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:28.242955 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:28.242957 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:28.243820 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:28.243820 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:28.244741 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:28.244742 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:28.245208 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:28.245209 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:28.245259 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:28.262851 (+ 17592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:28.262855 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:28.263180 (+ 325us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:52:28.264826 (+ 1646us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:28.264827 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:28.264827 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:28.265000 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:28.265170 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:28.265171 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:28.265171 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:28.265224 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:28.265466 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:52:28.265466 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:52:28.265505 (+ 39us) client_negotiation.cc:769] Sending connection context
0808 16:52:28.265873 (+ 368us) client_negotiation.cc:240] Negotiation successful
0808 16:52:28.265964 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"mutex_wait_us":48,"thread_start_us":107,"threads_started":1}
Aug 08 16:52:28 dist-test-slave-6zbq krb5kdc[17768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207947, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.24.62@KRBTEST.COM for oryx/127.8.24.60@KRBTEST.COM
I20260808 16:52:28.273270 19059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:28.267592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55774 (local address 127.8.24.60:39771)
0808 16:52:28.267640 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:28.267641 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:28.267648 (+ 7us) server_negotiation.cc:407] Connection header received
0808 16:52:28.267670 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:28.267671 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:28.267677 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:28.267717 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:28.268017 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:28.268484 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:28.269410 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:28.269573 (+ 163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:28.269620 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:28.271305 (+ 1685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:28.271306 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:28.271307 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:28.271312 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:28.272402 (+ 1090us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:28.272735 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:28.272736 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:28.272736 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:28.272776 (+ 40us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:28.272921 (+ 145us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:28.272921 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:28.272922 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:28.272996 (+ 74us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:28.273035 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:28.273177 (+ 142us) server_negotiation.cc:299] Negotiation successful
0808 16:52:28.273224 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 16:52:28.273381 18779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:28.267432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.60:39771 (local address 127.0.0.1:55774)
0808 16:52:28.267478 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:28.267488 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:28.267526 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:28.267737 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:28.267738 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:28.267908 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:28.267957 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:28.267959 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:28.268552 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:28.268553 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:28.269336 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:28.269339 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:28.269626 (+ 287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:28.269627 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:28.269694 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:28.270015 (+ 321us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:28.270019 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:28.271232 (+ 1213us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:52:28.272468 (+ 1236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:28.272469 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:28.272470 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:28.272675 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:28.272813 (+ 138us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:28.272814 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:28.272814 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:28.272870 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:28.273043 (+ 173us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:52:28.273044 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:52:28.273083 (+ 39us) client_negotiation.cc:769] Sending connection context
0808 16:52:28.273150 (+ 67us) client_negotiation.cc:240] Negotiation successful
0808 16:52:28.273238 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260808 16:52:28.273761 17840 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.8.24.60:39771: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "2a210ec3273f40d1b29270637e35af7b"
member_type: NON_VOTER
last_known_addr {
host: "127.8.24.60"
port: 39771
}
attrs {
promote: true
}
}
dest_uuid: "678b35993e9b49aca0f3ad2bba5edfbd"
cas_config_opid_index: 5
I20260808 16:52:28.273979 17840 consensus_queue.cc:237] T 00000000000000000000000000000000 P 678b35993e9b49aca0f3ad2bba5edfbd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "678b35993e9b49aca0f3ad2bba5edfbd" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 46193 } } peers { permanent_uuid: "ca479471edff409d957dde1bb92c4144" member_type: NON_VOTER last_known_addr { host: "127.8.24.61" port: 45755 } attrs { promote: true } } peers { permanent_uuid: "2a210ec3273f40d1b29270637e35af7b" member_type: NON_VOTER last_known_addr { host: "127.8.24.60" port: 39771 } attrs { promote: true } }
I20260808 16:52:28.274420 18784 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 678b35993e9b49aca0f3ad2bba5edfbd [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 2a210ec3273f40d1b29270637e35af7b (127.8.24.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "678b35993e9b49aca0f3ad2bba5edfbd" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 46193 } } peers { permanent_uuid: "ca479471edff409d957dde1bb92c4144" member_type: NON_VOTER last_known_addr { host: "127.8.24.61" port: 45755 } attrs { promote: true } } peers { permanent_uuid: "2a210ec3273f40d1b29270637e35af7b" member_type: NON_VOTER last_known_addr { host: "127.8.24.60" port: 39771 } attrs { promote: true } } }
I20260808 16:52:28.275337 18784 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.8.24.60:39771
I20260808 16:52:28.275408 18784 sys_catalog.cc:455] T 00000000000000000000000000000000 P 678b35993e9b49aca0f3ad2bba5edfbd [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "678b35993e9b49aca0f3ad2bba5edfbd" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "678b35993e9b49aca0f3ad2bba5edfbd" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 46193 } } peers { permanent_uuid: "ca479471edff409d957dde1bb92c4144" member_type: NON_VOTER last_known_addr { host: "127.8.24.61" port: 45755 } attrs { promote: true } } peers { permanent_uuid: "2a210ec3273f40d1b29270637e35af7b" member_type: NON_VOTER last_known_addr { host: "127.8.24.60" port: 39771 } attrs { promote: true } } }
I20260808 16:52:28.275472 18784 sys_catalog.cc:458] T 00000000000000000000000000000000 P 678b35993e9b49aca0f3ad2bba5edfbd [sys.catalog]: This master's current role is: LEADER
I20260808 16:52:28.275653 18928 master.cc:537] Master@127.8.24.60:39771 shutting down...
I20260808 16:52:28.275810 19008 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2a210ec3273f40d1b29270637e35af7b [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:52:28.276755 19008 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2a210ec3273f40d1b29270637e35af7b [term 1 FOLLOWER]: Refusing update from remote peer 678b35993e9b49aca0f3ad2bba5edfbd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260808 16:52:28.276986 18780 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 678b35993e9b49aca0f3ad2bba5edfbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a210ec3273f40d1b29270637e35af7b" member_type: NON_VOTER last_known_addr { host: "127.8.24.60" port: 39771 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 16:52:28.277542 17818 consensus_peers.cc:487] T 00000000000000000000000000000000 P 678b35993e9b49aca0f3ad2bba5edfbd -> Peer 2a210ec3273f40d1b29270637e35af7b (127.8.24.60:39771): Couldn't send request to peer 2a210ec3273f40d1b29270637e35af7b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260808 16:52:28.277654 18924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:28.274414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55318 (local address 127.8.24.61:45755)
0808 16:52:28.274445 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:28.274447 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:28.274494 (+ 47us) server_negotiation.cc:407] Connection header received
0808 16:52:28.274572 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:28.274573 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:28.274592 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:28.274661 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:52:28.274774 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:28.275192 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:28.276627 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:28.277159 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:28.277216 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:28.277287 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:28.277575 (+ 288us) server_negotiation.cc:299] Negotiation successful
0808 16:52:28.277606 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260808 16:52:28.277549 18779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:28.274373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.61:45755 (local address 127.0.0.1:55318)
0808 16:52:28.274452 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:28.274462 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:28.274511 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:28.274684 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:28.274685 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:28.274686 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:52:28.274715 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:28.274717 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:28.275256 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:28.275257 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:28.276533 (+ 1276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:28.276537 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:28.277324 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:28.277325 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:28.277384 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:28.277407 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 16:52:28.277454 (+ 47us) client_negotiation.cc:240] Negotiation successful
0808 16:52:28.277490 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66}
I20260808 16:52:28.278133 18860 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ca479471edff409d957dde1bb92c4144 [term 1 LEARNER]: Refusing update from remote peer 678b35993e9b49aca0f3ad2bba5edfbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260808 16:52:28.278316 18780 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 678b35993e9b49aca0f3ad2bba5edfbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca479471edff409d957dde1bb92c4144" member_type: NON_VOTER last_known_addr { host: "127.8.24.61" port: 45755 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260808 16:52:28.278844 18860 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ca479471edff409d957dde1bb92c4144 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 2a210ec3273f40d1b29270637e35af7b (127.8.24.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "678b35993e9b49aca0f3ad2bba5edfbd" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 46193 } } peers { permanent_uuid: "ca479471edff409d957dde1bb92c4144" member_type: NON_VOTER last_known_addr { host: "127.8.24.61" port: 45755 } attrs { promote: true } } peers { permanent_uuid: "2a210ec3273f40d1b29270637e35af7b" member_type: NON_VOTER last_known_addr { host: "127.8.24.60" port: 39771 } attrs { promote: true } } }
I20260808 16:52:28.279798 18909 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca479471edff409d957dde1bb92c4144 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "678b35993e9b49aca0f3ad2bba5edfbd" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "678b35993e9b49aca0f3ad2bba5edfbd" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 46193 } } peers { permanent_uuid: "ca479471edff409d957dde1bb92c4144" member_type: NON_VOTER last_known_addr { host: "127.8.24.61" port: 45755 } attrs { promote: true } } peers { permanent_uuid: "2a210ec3273f40d1b29270637e35af7b" member_type: NON_VOTER last_known_addr { host: "127.8.24.60" port: 39771 } attrs { promote: true } } }
I20260808 16:52:28.279899 18909 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca479471edff409d957dde1bb92c4144 [sys.catalog]: This master's current role is: LEARNER
I20260808 16:52:28.279789 19095 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca479471edff409d957dde1bb92c4144 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 678b35993e9b49aca0f3ad2bba5edfbd. Latest consensus state: current_term: 1 leader_uuid: "678b35993e9b49aca0f3ad2bba5edfbd" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "678b35993e9b49aca0f3ad2bba5edfbd" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 46193 } } peers { permanent_uuid: "ca479471edff409d957dde1bb92c4144" member_type: NON_VOTER last_known_addr { host: "127.8.24.61" port: 45755 } attrs { promote: true } } peers { permanent_uuid: "2a210ec3273f40d1b29270637e35af7b" member_type: NON_VOTER last_known_addr { host: "127.8.24.60" port: 39771 } attrs { promote: true } } }
I20260808 16:52:28.279976 19095 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca479471edff409d957dde1bb92c4144 [sys.catalog]: This master's current role is: LEARNER
I20260808 16:52:28.280544 18780 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 678b35993e9b49aca0f3ad2bba5edfbd: attempting to promote NON_VOTER ca479471edff409d957dde1bb92c4144 to VOTER
I20260808 16:52:28.280593 18928 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 2a210ec3273f40d1b29270637e35af7b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 16:52:28.280668 18928 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 2a210ec3273f40d1b29270637e35af7b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:52:28.280694 18928 tablet_replica.cc:330] T 00000000000000000000000000000000 P 2a210ec3273f40d1b29270637e35af7b: stopping tablet replica
I20260808 16:52:28.280727 18780 consensus_queue.cc:237] T 00000000000000000000000000000000 P 678b35993e9b49aca0f3ad2bba5edfbd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "678b35993e9b49aca0f3ad2bba5edfbd" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 46193 } } peers { permanent_uuid: "ca479471edff409d957dde1bb92c4144" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 45755 } attrs { promote: false } } peers { permanent_uuid: "2a210ec3273f40d1b29270637e35af7b" member_type: NON_VOTER last_known_addr { host: "127.8.24.60" port: 39771 } attrs { promote: true } }
I20260808 16:52:28.281452 18860 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ca479471edff409d957dde1bb92c4144 [term 1 LEARNER]: Refusing update from remote peer 678b35993e9b49aca0f3ad2bba5edfbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
W20260808 16:52:28.281513 17818 consensus_peers.cc:487] T 00000000000000000000000000000000 P 678b35993e9b49aca0f3ad2bba5edfbd -> Peer 2a210ec3273f40d1b29270637e35af7b (127.8.24.60:39771): Couldn't send request to peer 2a210ec3273f40d1b29270637e35af7b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
W20260808 16:52:28.281605 17766 connection.cc:535] client connection to 127.8.24.60:39771 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 16:52:28.281764 18782 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 678b35993e9b49aca0f3ad2bba5edfbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca479471edff409d957dde1bb92c4144" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 45755 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260808 16:52:28.282490 18782 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 678b35993e9b49aca0f3ad2bba5edfbd [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, ca479471edff409d957dde1bb92c4144 (127.8.24.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "678b35993e9b49aca0f3ad2bba5edfbd" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 46193 } } peers { permanent_uuid: "ca479471edff409d957dde1bb92c4144" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 45755 } attrs { promote: false } } peers { permanent_uuid: "2a210ec3273f40d1b29270637e35af7b" member_type: NON_VOTER last_known_addr { host: "127.8.24.60" port: 39771 } attrs { promote: true } } }
I20260808 16:52:28.282668 18860 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ca479471edff409d957dde1bb92c4144 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, ca479471edff409d957dde1bb92c4144 (127.8.24.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "678b35993e9b49aca0f3ad2bba5edfbd" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 46193 } } peers { permanent_uuid: "ca479471edff409d957dde1bb92c4144" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 45755 } attrs { promote: false } } peers { permanent_uuid: "2a210ec3273f40d1b29270637e35af7b" member_type: NON_VOTER last_known_addr { host: "127.8.24.60" port: 39771 } attrs { promote: true } } }
I20260808 16:52:28.283250 18782 sys_catalog.cc:455] T 00000000000000000000000000000000 P 678b35993e9b49aca0f3ad2bba5edfbd [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "678b35993e9b49aca0f3ad2bba5edfbd" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "678b35993e9b49aca0f3ad2bba5edfbd" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 46193 } } peers { permanent_uuid: "ca479471edff409d957dde1bb92c4144" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 45755 } attrs { promote: false } } peers { permanent_uuid: "2a210ec3273f40d1b29270637e35af7b" member_type: NON_VOTER last_known_addr { host: "127.8.24.60" port: 39771 } attrs { promote: true } } }
I20260808 16:52:28.283330 18782 sys_catalog.cc:458] T 00000000000000000000000000000000 P 678b35993e9b49aca0f3ad2bba5edfbd [sys.catalog]: This master's current role is: LEADER
I20260808 16:52:28.283376 18782 sys_catalog.cc:455] T 00000000000000000000000000000000 P 678b35993e9b49aca0f3ad2bba5edfbd [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "678b35993e9b49aca0f3ad2bba5edfbd" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "678b35993e9b49aca0f3ad2bba5edfbd" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 46193 } } peers { permanent_uuid: "ca479471edff409d957dde1bb92c4144" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 45755 } attrs { promote: false } } peers { permanent_uuid: "2a210ec3273f40d1b29270637e35af7b" member_type: NON_VOTER last_known_addr { host: "127.8.24.60" port: 39771 } attrs { promote: true } } }
I20260808 16:52:28.283423 18782 sys_catalog.cc:458] T 00000000000000000000000000000000 P 678b35993e9b49aca0f3ad2bba5edfbd [sys.catalog]: This master's current role is: LEADER
I20260808 16:52:28.283416 19095 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca479471edff409d957dde1bb92c4144 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "678b35993e9b49aca0f3ad2bba5edfbd" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "678b35993e9b49aca0f3ad2bba5edfbd" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 46193 } } peers { permanent_uuid: "ca479471edff409d957dde1bb92c4144" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 45755 } attrs { promote: false } } peers { permanent_uuid: "2a210ec3273f40d1b29270637e35af7b" member_type: NON_VOTER last_known_addr { host: "127.8.24.60" port: 39771 } attrs { promote: true } } }
I20260808 16:52:28.283519 19095 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca479471edff409d957dde1bb92c4144 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 16:52:28.293260 18928 master.cc:552] Master@127.8.24.60:39771 shutdown complete.
I20260808 16:52:28.293305 18928 master_runner.cc:305] Clearing existing system tablet
I20260808 16:52:28.293413 18928 fs_manager.cc:359] Metadata directory not provided
I20260808 16:52:28.293458 18928 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:52:28.294106 18928 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:28.294664 19098 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:28.294804 18928 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:52:28.294863 18928 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:28.294888 18928 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/master-2/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/master-2/wal
uuid: "2a210ec3273f40d1b29270637e35af7b"
format_stamp: "Formatted at 2026-08-08 16:52:28 on dist-test-slave-6zbq"
server_key: "27363af375b22d2d447906e0d11fabe2"
server_key_iv: "51342327929f33f9b3867ff39a965cb5"
server_key_version: "encryptionkey@0"
I20260808 16:52:28.295547 18928 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 2a210ec3273f40d1b29270637e35af7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:52:28.296725 18928 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 2a210ec3273f40d1b29270637e35af7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260808 16:52:28.296792 18928 log.cc:1196] T 00000000000000000000000000000000 P 2a210ec3273f40d1b29270637e35af7b: Deleting WAL directory at /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260808 16:52:28.296968 18928 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 2a210ec3273f40d1b29270637e35af7b: Deleting consensus metadata
I20260808 16:52:28.297086 18928 master_runner.cc:315] Copying system tablet from 127.8.24.62:46193
I20260808 16:52:28.297619 18928 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 2a210ec3273f40d1b29270637e35af7b: tablet copy: Beginning tablet copy session from remote peer at address 127.8.24.62:46193
I20260808 16:52:28.303105 17917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:28.297904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59024 (local address 127.8.24.62:46193)
0808 16:52:28.297943 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:28.297945 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:28.297997 (+ 52us) server_negotiation.cc:407] Connection header received
0808 16:52:28.298061 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:28.298061 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:28.298068 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:28.298108 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:28.298457 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:28.298887 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:28.299884 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:28.300045 (+ 161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:28.300083 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:28.300896 (+ 813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:28.300897 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:28.300898 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:28.300903 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:28.302026 (+ 1123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:28.302417 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:28.302418 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:28.302419 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:28.302491 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:28.302679 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:28.302680 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:28.302680 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:28.302773 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:28.302825 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:28.302959 (+ 134us) server_negotiation.cc:299] Negotiation successful
0808 16:52:28.303021 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 16:52:28.303125 19109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:28.297838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:46193 (local address 127.0.0.1:59024)
0808 16:52:28.297944 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:28.297953 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:28.298010 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:28.298114 (+ 104us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:28.298115 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:28.298284 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:28.298333 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:28.298335 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:28.298956 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:28.298957 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:28.299816 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:28.299818 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:28.300097 (+ 279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:28.300098 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:28.300150 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:28.300485 (+ 335us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:28.300489 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:28.300823 (+ 334us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:52:28.302101 (+ 1278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:28.302102 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:28.302102 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:28.302305 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:28.302548 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:28.302549 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:28.302549 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:28.302631 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:28.302828 (+ 197us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:52:28.302829 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:52:28.302870 (+ 41us) client_negotiation.cc:769] Sending connection context
0808 16:52:28.302928 (+ 58us) client_negotiation.cc:240] Negotiation successful
0808 16:52:28.303022 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87,"thread_start_us":69,"threads_started":1}
I20260808 16:52:28.303300 17878 tablet_copy_service.cc:140] P 678b35993e9b49aca0f3ad2bba5edfbd: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 2a210ec3273f40d1b29270637e35af7b ({username='oryx', principal='oryx/127.8.24.60@KRBTEST.COM'} at 127.0.0.1:59024)
I20260808 16:52:28.303351 17878 tablet_copy_service.cc:161] P 678b35993e9b49aca0f3ad2bba5edfbd: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 2a210ec3273f40d1b29270637e35af7b at {username='oryx', principal='oryx/127.8.24.60@KRBTEST.COM'} at 127.0.0.1:59024: session id = 2a210ec3273f40d1b29270637e35af7b-00000000000000000000000000000000
I20260808 16:52:28.303795 17878 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 678b35993e9b49aca0f3ad2bba5edfbd: Tablet Copy: opened 0 blocks and 1 log segments
I20260808 16:52:28.304224 18928 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260808 16:52:28.305380 18928 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 2a210ec3273f40d1b29270637e35af7b: tablet copy: Starting download of 0 data blocks...
I20260808 16:52:28.305523 18928 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 2a210ec3273f40d1b29270637e35af7b: tablet copy: Starting download of 1 WAL segments...
I20260808 16:52:28.306214 18928 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 2a210ec3273f40d1b29270637e35af7b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260808 16:52:28.306949 17878 tablet_copy_service.cc:342] P 678b35993e9b49aca0f3ad2bba5edfbd: Request end of tablet copy session 2a210ec3273f40d1b29270637e35af7b-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.8.24.60@KRBTEST.COM'} at 127.0.0.1:59024
I20260808 16:52:28.306993 17878 tablet_copy_service.cc:433] P 678b35993e9b49aca0f3ad2bba5edfbd: ending tablet copy session 2a210ec3273f40d1b29270637e35af7b-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 2a210ec3273f40d1b29270637e35af7b
I20260808 16:52:28.309125 18928 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:52:28.310046 19112 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:28.310117 19110 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:28.311179 19115 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:28.311219 18928 server_base.cc:1034] running on GCE node
I20260808 16:52:28.311388 18928 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:52:28.311508 18928 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:52:28.312618 18928 hybrid_clock.cc:648] HybridClock initialized: now 1786207948312608 us; error 27 us; skew 500 ppm
Aug 08 16:52:28 dist-test-slave-6zbq krb5kdc[17768](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207948, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.24.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:52:28.313813 18928 init.cc:377] Logged in from keytab as oryx/127.8.24.60@KRBTEST.COM (short username oryx)
I20260808 16:52:28.314067 18928 webserver.cc:460] Webserver started at http://127.8.24.60:44781/ using document root <none> and password file <none>
I20260808 16:52:28.314177 18928 fs_manager.cc:359] Metadata directory not provided
I20260808 16:52:28.314214 18928 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:52:28.314909 18928 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:28.315408 19123 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:28.315546 18928 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:28.315600 18928 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/master-2/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/master-2/wal
uuid: "2a210ec3273f40d1b29270637e35af7b"
format_stamp: "Formatted at 2026-08-08 16:52:28 on dist-test-slave-6zbq"
server_key: "27363af375b22d2d447906e0d11fabe2"
server_key_iv: "51342327929f33f9b3867ff39a965cb5"
server_key_version: "encryptionkey@0"
I20260808 16:52:28.315661 18928 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:52:28.323223 18928 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:52:28.324566 18928 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:52:28.327628 18928 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.60:39771
I20260808 16:52:28.327790 19200 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.60:39771 every 8 connection(s)
I20260808 16:52:28.327972 18928 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/master-2/data/info.pb
I20260808 16:52:28.328560 19205 sys_catalog.cc:263] Verifying existing consensus state
I20260808 16:52:28.328922 19205 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a210ec3273f40d1b29270637e35af7b: Bootstrap starting.
I20260808 16:52:28.331089 19205 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a210ec3273f40d1b29270637e35af7b: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 16:52:28.331277 19205 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a210ec3273f40d1b29270637e35af7b: Bootstrap complete.
I20260808 16:52:28.331452 19205 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2a210ec3273f40d1b29270637e35af7b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "678b35993e9b49aca0f3ad2bba5edfbd" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 46193 } } peers { permanent_uuid: "ca479471edff409d957dde1bb92c4144" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 45755 } attrs { promote: false } } peers { permanent_uuid: "2a210ec3273f40d1b29270637e35af7b" member_type: NON_VOTER last_known_addr { host: "127.8.24.60" port: 39771 } attrs { promote: true } }
I20260808 16:52:28.331528 19205 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2a210ec3273f40d1b29270637e35af7b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 2a210ec3273f40d1b29270637e35af7b, State: Initialized, Role: LEARNER
I20260808 16:52:28.331573 19205 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2a210ec3273f40d1b29270637e35af7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "678b35993e9b49aca0f3ad2bba5edfbd" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 46193 } } peers { permanent_uuid: "ca479471edff409d957dde1bb92c4144" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 45755 } attrs { promote: false } } peers { permanent_uuid: "2a210ec3273f40d1b29270637e35af7b" member_type: NON_VOTER last_known_addr { host: "127.8.24.60" port: 39771 } attrs { promote: true } }
I20260808 16:52:28.331744 19205 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2a210ec3273f40d1b29270637e35af7b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 16:52:28.331729 19209 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a210ec3273f40d1b29270637e35af7b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "678b35993e9b49aca0f3ad2bba5edfbd" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 46193 } } peers { permanent_uuid: "ca479471edff409d957dde1bb92c4144" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 45755 } attrs { promote: false } } peers { permanent_uuid: "2a210ec3273f40d1b29270637e35af7b" member_type: NON_VOTER last_known_addr { host: "127.8.24.60" port: 39771 } attrs { promote: true } } }
I20260808 16:52:28.331943 19209 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a210ec3273f40d1b29270637e35af7b [sys.catalog]: This master's current role is: LEARNER
I20260808 16:52:28.333082 19228 catalog_manager.cc:1259] Loaded cluster ID: ddde3755643b4afbb76aa4f0f8896bc8
I20260808 16:52:28.333163 19228 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 2a210ec3273f40d1b29270637e35af7b: loading cluster ID for follower catalog manager: success
I20260808 16:52:28.334398 19228 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 2a210ec3273f40d1b29270637e35af7b: acquiring CA information for follower catalog manager: success
I20260808 16:52:28.334682 19228 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 2a210ec3273f40d1b29270637e35af7b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 16:52:28.339056 19215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:28.332151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55786 (local address 127.8.24.60:39771)
0808 16:52:28.332289 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:28.332291 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:28.332299 (+ 8us) server_negotiation.cc:407] Connection header received
0808 16:52:28.332320 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:28.332320 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:28.332331 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:28.333062 (+ 731us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:28.333197 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:28.334543 (+ 1346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:28.335263 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:28.335580 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:28.335648 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:28.336633 (+ 985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:28.336634 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:28.336635 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:28.336642 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:28.338032 (+ 1390us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:28.338332 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:28.338333 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:28.338334 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:28.338426 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:28.338591 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:28.338591 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:28.338592 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:28.338720 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:28.338815 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:28.338878 (+ 63us) server_negotiation.cc:299] Negotiation successful
0808 16:52:28.338966 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":90,"threads_started":1}
I20260808 16:52:28.339488 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 17954
I20260808 16:52:28.355312 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.1:38887
--local_ip_for_outbound_sockets=127.8.24.1
--tserver_master_addrs=127.8.24.62:46193
--webserver_port=33339
--webserver_interface=127.8.24.1
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/krb5kdc/oryx_127.8.24.1.keytab
--principal=oryx/127.8.24.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:37767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.8.24.62:46193,127.8.24.61:45755
--tserver_master_addrs=127.8.24.62:46193,127.8.24.61:45755,127.8.24.60:39771 with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:52:28.446671 19232 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:52:28.446841 19232 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:52:28.446861 19232 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:52:28.447964 19232 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:52:28.448009 19232 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:52:28.448043 19232 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.1
I20260808 16:52:28.449493 19232 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:37767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/krb5kdc/oryx_127.8.24.1.keytab
--principal=oryx/127.8.24.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.1:38887
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.24.1
--webserver_port=33339
--tserver_master_addrs=127.8.24.62:46193,127.8.24.61:45755,127.8.24.60:39771
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19232
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.1
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:52:28.449721 19232 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:52:28.449944 19232 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:52:28.454428 19238 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:28.454438 19241 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:28.454514 19232 server_base.cc:1034] running on GCE node
W20260808 16:52:28.454427 19244 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:28.454799 19232 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:52:28.455011 19232 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:52:28.456146 19232 hybrid_clock.cc:648] HybridClock initialized: now 1786207948456134 us; error 30 us; skew 500 ppm
Aug 08 16:52:28 dist-test-slave-6zbq krb5kdc[17768](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207948, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.24.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:52:28.458312 19232 init.cc:377] Logged in from keytab as oryx/127.8.24.1@KRBTEST.COM (short username oryx)
I20260808 16:52:28.458747 19232 webserver.cc:460] Webserver started at http://127.8.24.1:33339/ using document root <none> and password file <none>
I20260808 16:52:28.458933 19232 fs_manager.cc:359] Metadata directory not provided
I20260808 16:52:28.458992 19232 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:52:28.460184 19232 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 16:52:28.460706 19258 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:28.460846 19232 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 16:52:28.460922 19232 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/ts-0/wal
uuid: "7b46262135f64e30b03b7b7cfa5e7044"
format_stamp: "Formatted at 2026-08-08 16:52:27 on dist-test-slave-6zbq"
server_key: "bed61d00eef2026f4dbe6727862f0cab"
server_key_iv: "bece949b1fa06c287a8a8e977305c2af"
server_key_version: "encryptionkey@0"
I20260808 16:52:28.461227 19232 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:52:28.479470 19232 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:52:28.481017 19232 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:52:28.481161 19232 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:52:28.481348 19232 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:52:28.481606 19232 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:52:28.481637 19232 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:28.481663 19232 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:52:28.481680 19232 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:28.487056 19232 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.1:38887
I20260808 16:52:28.487092 19438 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.1:38887 every 8 connection(s)
I20260808 16:52:28.487386 19232 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/ts-0/data/info.pb
I20260808 16:52:28.490758 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 19232
I20260808 16:52:28.490895 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 18187
Aug 08 16:52:28 dist-test-slave-6zbq krb5kdc[17768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207948, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.24.1@KRBTEST.COM for oryx/127.8.24.61@KRBTEST.COM
Aug 08 16:52:28 dist-test-slave-6zbq krb5kdc[17768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207948, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.24.1@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM
Aug 08 16:52:28 dist-test-slave-6zbq krb5kdc[17768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207948, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.24.1@KRBTEST.COM for oryx/127.8.24.60@KRBTEST.COM
I20260808 16:52:28.504737 17917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:28.488026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:39913 (local address 127.8.24.62:46193)
0808 16:52:28.488074 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:28.488076 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:28.488250 (+ 174us) server_negotiation.cc:407] Connection header received
0808 16:52:28.489181 (+ 931us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:28.489182 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:28.489194 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:28.489254 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:28.489995 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:28.490797 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:28.491780 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:28.492076 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:28.492145 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:28.497273 (+ 5128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:28.497275 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:28.497275 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:28.497282 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:28.501310 (+ 4028us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:28.503522 (+ 2212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:28.503524 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:28.503525 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:28.503608 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:28.503875 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:28.503876 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:28.503876 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:28.504026 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:28.504109 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:28.504513 (+ 404us) server_negotiation.cc:299] Negotiation successful
0808 16:52:28.504627 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 16:52:28.505839 19448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:28.487952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:46193 (local address 127.8.24.1:39913)
0808 16:52:28.488125 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:28.488161 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:28.489081 (+ 920us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:28.489324 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:28.489327 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:28.489721 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:28.489906 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:28.489911 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:28.490893 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:28.490894 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:28.491698 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:28.491701 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:28.492209 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:28.492210 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:28.492338 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:28.493606 (+ 1268us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:28.493615 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:28.497157 (+ 3542us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:52:28.502339 (+ 5182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:28.502517 (+ 178us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:28.502523 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:28.503430 (+ 907us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:28.503672 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:28.503673 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:28.503673 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:28.503820 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:28.504151 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:52:28.504159 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:52:28.504354 (+ 195us) client_negotiation.cc:769] Sending connection context
0808 16:52:28.505295 (+ 941us) client_negotiation.cc:240] Negotiation successful
0808 16:52:28.505567 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"mutex_wait_us":37,"thread_start_us":88,"threads_started":1}
I20260808 16:52:28.506979 19440 heartbeater.cc:347] Connected to a master server at 127.8.24.62:46193
I20260808 16:52:28.507097 19440 heartbeater.cc:464] Registering TS with master...
I20260808 16:52:28.507346 19440 heartbeater.cc:511] Master 127.8.24.62:46193 requested a full tablet report, sending...
I20260808 16:52:28.507494 19215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:28.488295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:36759 (local address 127.8.24.60:39771)
0808 16:52:28.488338 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:28.488339 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:28.489299 (+ 960us) server_negotiation.cc:407] Connection header received
0808 16:52:28.489357 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:28.489357 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:28.489365 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:28.489401 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:28.490037 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:28.490804 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:28.492523 (+ 1719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:28.492732 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:28.492764 (+ 32us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:28.501129 (+ 8365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:28.501130 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:28.501131 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:28.501140 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:28.505615 (+ 4475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:28.506235 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:28.506236 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:28.506237 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:28.506325 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:28.506637 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:28.506638 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:28.506638 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:28.506835 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:28.506912 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:28.507241 (+ 329us) server_negotiation.cc:299] Negotiation successful
0808 16:52:28.507366 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 16:52:28.507767 19445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:28.488215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.60:39771 (local address 127.8.24.1:36759)
0808 16:52:28.489250 (+ 1035us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:28.489259 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:28.489306 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:28.489407 (+ 101us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:28.489407 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:28.489712 (+ 305us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:28.489906 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:28.489912 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:28.490914 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:28.490915 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:28.492425 (+ 1510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:28.492426 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:28.492823 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:28.492824 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:28.492868 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:28.494305 (+ 1437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:28.494310 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:28.499028 (+ 4718us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:52:28.505770 (+ 6742us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:28.505772 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:28.505772 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:28.506079 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:28.506431 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:28.506431 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:28.506432 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:28.506544 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:28.506985 (+ 441us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:52:28.506986 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:52:28.507037 (+ 51us) client_negotiation.cc:769] Sending connection context
0808 16:52:28.507589 (+ 552us) client_negotiation.cc:240] Negotiation successful
0808 16:52:28.507683 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1020,"mutex_wait_us":20,"thread_start_us":58,"threads_started":1}
I20260808 16:52:28.507802 17840 ts_manager.cc:194] Re-registered known tserver with Master: 7b46262135f64e30b03b7b7cfa5e7044 (127.8.24.1:38887)
I20260808 16:52:28.508322 17840 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.24.1@KRBTEST.COM'} at 127.8.24.1:39913
I20260808 16:52:28.508580 19439 heartbeater.cc:347] Connected to a master server at 127.8.24.60:39771
I20260808 16:52:28.508630 19439 heartbeater.cc:464] Registering TS with master...
I20260808 16:52:28.508709 19439 heartbeater.cc:511] Master 127.8.24.60:39771 requested a full tablet report, sending...
I20260808 16:52:28.510046 18924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:28.487962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:41499 (local address 127.8.24.61:45755)
0808 16:52:28.488016 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:28.488019 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:28.488364 (+ 345us) server_negotiation.cc:407] Connection header received
0808 16:52:28.489181 (+ 817us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:28.489182 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:28.489194 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:28.489254 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:28.490043 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:28.490556 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:28.491990 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:28.492376 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:28.492425 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:28.504810 (+ 12385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:28.504812 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:28.504813 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:28.504825 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:28.506756 (+ 1931us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:28.509359 (+ 2603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:28.509361 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:28.509361 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:28.509419 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:28.509568 (+ 149us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:28.509569 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:28.509569 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:28.509681 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:28.509712 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:28.509888 (+ 176us) server_negotiation.cc:299] Negotiation successful
0808 16:52:28.509960 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 16:52:28.510046 19450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:28.487953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.61:45755 (local address 127.8.24.1:41499)
0808 16:52:28.488313 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:28.488322 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:28.489092 (+ 770us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:28.489547 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:28.489547 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:28.489735 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:28.489906 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:28.489912 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:28.490693 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:28.490694 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:28.491873 (+ 1179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:28.491876 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:28.492510 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:28.492511 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:28.492583 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:28.493753 (+ 1170us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:28.493758 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:28.499655 (+ 5897us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:52:28.509091 (+ 9436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:28.509091 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:28.509092 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:28.509278 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:28.509472 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:28.509473 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:28.509473 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:28.509515 (+ 42us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:28.509744 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:52:28.509745 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:52:28.509775 (+ 30us) client_negotiation.cc:769] Sending connection context
0808 16:52:28.509838 (+ 63us) client_negotiation.cc:240] Negotiation successful
0808 16:52:28.509945 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":336,"mutex_wait_us":2,"thread_start_us":92,"threads_started":1}
I20260808 16:52:28.510110 19150 ts_manager.cc:194] Registered new tserver with Master: 7b46262135f64e30b03b7b7cfa5e7044 (127.8.24.1:38887)
I20260808 16:52:28.510426 19441 heartbeater.cc:347] Connected to a master server at 127.8.24.61:45755
I20260808 16:52:28.510480 19441 heartbeater.cc:464] Registering TS with master...
I20260808 16:52:28.510537 19441 heartbeater.cc:511] Master 127.8.24.61:45755 requested a full tablet report, sending...
I20260808 16:52:28.510874 18847 ts_manager.cc:194] Registered new tserver with Master: 7b46262135f64e30b03b7b7cfa5e7044 (127.8.24.1:38887)
I20260808 16:52:28.513958 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.2:38759
--local_ip_for_outbound_sockets=127.8.24.2
--tserver_master_addrs=127.8.24.62:46193
--webserver_port=41471
--webserver_interface=127.8.24.2
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/krb5kdc/oryx_127.8.24.2.keytab
--principal=oryx/127.8.24.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:37767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.8.24.62:46193,127.8.24.61:45755
--tserver_master_addrs=127.8.24.62:46193,127.8.24.61:45755,127.8.24.60:39771 with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:52:28.594017 19454 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:52:28.594187 19454 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:52:28.594208 19454 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:52:28.595276 19454 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:52:28.595314 19454 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:52:28.595337 19454 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.2
I20260808 16:52:28.596671 19454 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:37767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/krb5kdc/oryx_127.8.24.2.keytab
--principal=oryx/127.8.24.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.2:38759
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.24.2
--webserver_port=41471
--tserver_master_addrs=127.8.24.62:46193,127.8.24.61:45755,127.8.24.60:39771
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19454
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.2
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:52:28.596865 19454 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:52:28.597105 19454 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:52:28.601833 19461 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:28.601840 19465 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:28.601856 19460 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:28.601855 19454 server_base.cc:1034] running on GCE node
I20260808 16:52:28.602175 19454 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:52:28.602424 19454 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:52:28.603576 19454 hybrid_clock.cc:648] HybridClock initialized: now 1786207948603554 us; error 31 us; skew 500 ppm
Aug 08 16:52:28 dist-test-slave-6zbq krb5kdc[17768](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207948, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.24.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:52:28.606578 19454 init.cc:377] Logged in from keytab as oryx/127.8.24.2@KRBTEST.COM (short username oryx)
I20260808 16:52:28.607020 19454 webserver.cc:460] Webserver started at http://127.8.24.2:41471/ using document root <none> and password file <none>
I20260808 16:52:28.607255 19454 fs_manager.cc:359] Metadata directory not provided
I20260808 16:52:28.607324 19454 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:52:28.608935 19454 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 16:52:28.609591 19477 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:28.609810 19454 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 16:52:28.609903 19454 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/ts-1/wal
uuid: "005e71fe78a847c2b010b116c89a8609"
format_stamp: "Formatted at 2026-08-08 16:52:27 on dist-test-slave-6zbq"
server_key: "a20a77b1d9a353bcd84c52447d3c3e00"
server_key_iv: "f7bcfc50abde4a3f92452e69b76b9e4b"
server_key_version: "encryptionkey@0"
I20260808 16:52:28.610275 19454 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:52:28.640729 19454 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:52:28.642700 19454 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:52:28.642807 19454 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:52:28.642973 19454 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:52:28.643239 19454 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:52:28.643272 19454 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:28.643291 19454 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:52:28.643314 19454 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:28.648624 19454 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.2:38759
I20260808 16:52:28.648699 19657 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.2:38759 every 8 connection(s)
I20260808 16:52:28.649056 19454 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/ts-1/data/info.pb
Aug 08 16:52:28 dist-test-slave-6zbq krb5kdc[17768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207948, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.24.2@KRBTEST.COM for oryx/127.8.24.61@KRBTEST.COM
Aug 08 16:52:28 dist-test-slave-6zbq krb5kdc[17768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207948, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.24.2@KRBTEST.COM for oryx/127.8.24.60@KRBTEST.COM
Aug 08 16:52:28 dist-test-slave-6zbq krb5kdc[17768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207948, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.24.2@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM
I20260808 16:52:28.658797 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 19454
I20260808 16:52:28.658948 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 18399
I20260808 16:52:28.659317 19215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:28.650073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:42665 (local address 127.8.24.60:39771)
0808 16:52:28.650129 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:28.650131 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:28.650295 (+ 164us) server_negotiation.cc:407] Connection header received
0808 16:52:28.651042 (+ 747us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:28.651044 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:28.651057 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:28.651087 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:28.652427 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:28.653142 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:28.653844 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:28.654039 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:28.654093 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:28.656617 (+ 2524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:28.656618 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:28.656619 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:28.656626 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:28.657971 (+ 1345us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:28.658328 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:28.658329 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:28.658330 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:28.658404 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:28.658607 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:28.658608 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:28.658608 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:28.658763 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:28.658823 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:28.659151 (+ 328us) server_negotiation.cc:299] Negotiation successful
0808 16:52:28.659229 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 16:52:28.659701 19664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:28.649991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.60:39771 (local address 127.8.24.2:42665)
0808 16:52:28.650242 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:28.650250 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:28.650927 (+ 677us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:28.651204 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:28.651208 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:28.651587 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:28.651761 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:28.651767 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:28.653221 (+ 1454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:28.653222 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:28.653767 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:28.653769 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:28.654097 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:28.654098 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:28.654211 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:28.654620 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:28.654629 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:28.656531 (+ 1902us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:52:28.658041 (+ 1510us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:28.658047 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:28.658049 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:28.658262 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:28.658458 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:28.658459 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:28.658459 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:28.658552 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:28.658852 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:52:28.658859 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:52:28.659008 (+ 149us) client_negotiation.cc:769] Sending connection context
0808 16:52:28.659392 (+ 384us) client_negotiation.cc:240] Negotiation successful
0808 16:52:28.659531 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227,"mutex_wait_us":4,"thread_start_us":56,"threads_started":1}
I20260808 16:52:28.661878 18924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:28.649667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:53701 (local address 127.8.24.61:45755)
0808 16:52:28.649714 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:28.649716 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:28.649986 (+ 270us) server_negotiation.cc:407] Connection header received
0808 16:52:28.651021 (+ 1035us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:28.651022 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:28.651035 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:28.651082 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:28.651849 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:28.652327 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:28.653683 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:28.653970 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:28.654007 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:28.657009 (+ 3002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:28.657010 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:28.657011 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:28.657018 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:28.658532 (+ 1514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:28.658915 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:28.658917 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:28.658917 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:28.658972 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:28.661438 (+ 2466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:28.661439 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:28.661440 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:28.661545 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:28.661577 (+ 32us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:28.661744 (+ 167us) server_negotiation.cc:299] Negotiation successful
0808 16:52:28.661808 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 16:52:28.662093 19663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:28.649695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.61:45755 (local address 127.8.24.2:53701)
0808 16:52:28.649875 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:28.649909 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:28.650918 (+ 1009us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:28.651205 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:28.651209 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:28.651570 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:28.651760 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:28.651765 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:28.652425 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:28.652426 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:28.653592 (+ 1166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:28.653594 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:28.654087 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:28.654088 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:28.654213 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:28.654760 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:28.654765 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:28.656906 (+ 2141us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:52:28.658616 (+ 1710us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:28.658618 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:28.658618 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:28.658827 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:28.661319 (+ 2492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:28.661320 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:28.661320 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:28.661378 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:28.661608 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:52:28.661609 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:52:28.661640 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 16:52:28.661943 (+ 303us) client_negotiation.cc:240] Negotiation successful
0808 16:52:28.662017 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":105,"threads_started":1}
I20260808 16:52:28.662947 19660 heartbeater.cc:347] Connected to a master server at 127.8.24.61:45755
I20260808 16:52:28.663038 19660 heartbeater.cc:464] Registering TS with master...
I20260808 16:52:28.663220 19660 heartbeater.cc:511] Master 127.8.24.61:45755 requested a full tablet report, sending...
I20260808 16:52:28.663704 18847 ts_manager.cc:194] Registered new tserver with Master: 005e71fe78a847c2b010b116c89a8609 (127.8.24.2:38759)
I20260808 16:52:28.666282 19665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:28.649695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:46193 (local address 127.8.24.2:43051)
0808 16:52:28.650137 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:28.650147 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:28.650924 (+ 777us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:28.651204 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:28.651208 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:28.651594 (+ 386us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:28.651761 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:28.651767 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:28.654020 (+ 2253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:28.654021 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:28.654719 (+ 698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:28.654723 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:28.655227 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:28.655228 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:28.655290 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:28.655637 (+ 347us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:28.655640 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:28.657028 (+ 1388us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:52:28.665372 (+ 8344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:28.665373 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:28.665373 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:28.665539 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:28.665713 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:28.665714 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:28.665714 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:28.665769 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:28.665977 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:52:28.665978 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:52:28.666018 (+ 40us) client_negotiation.cc:769] Sending connection context
0808 16:52:28.666073 (+ 55us) client_negotiation.cc:240] Negotiation successful
0808 16:52:28.666160 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":415,"mutex_wait_us":21,"thread_start_us":105,"threads_started":1}
I20260808 16:52:28.666285 17917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:28.649703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:43051 (local address 127.8.24.62:46193)
0808 16:52:28.649750 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:28.649752 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:28.650213 (+ 461us) server_negotiation.cc:407] Connection header received
0808 16:52:28.651075 (+ 862us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:28.651076 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:28.651088 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:28.651138 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:28.653241 (+ 2103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:28.653941 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:28.654805 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:28.655096 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:28.655163 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:28.657131 (+ 1968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:28.657132 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:28.657133 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:28.657139 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:28.659111 (+ 1972us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:28.665604 (+ 6493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:28.665605 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:28.665605 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:28.665661 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:28.665825 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:28.665826 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:28.665826 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:28.665928 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:28.665980 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:28.666118 (+ 138us) server_negotiation.cc:299] Negotiation successful
0808 16:52:28.666196 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 16:52:28.666996 19659 heartbeater.cc:347] Connected to a master server at 127.8.24.62:46193
I20260808 16:52:28.667042 19659 heartbeater.cc:464] Registering TS with master...
I20260808 16:52:28.667101 19659 heartbeater.cc:511] Master 127.8.24.62:46193 requested a full tablet report, sending...
I20260808 16:52:28.667346 17840 ts_manager.cc:194] Re-registered known tserver with Master: 005e71fe78a847c2b010b116c89a8609 (127.8.24.2:38759)
I20260808 16:52:28.667958 17840 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.24.2@KRBTEST.COM'} at 127.8.24.2:43051
I20260808 16:52:28.683400 19658 heartbeater.cc:347] Connected to a master server at 127.8.24.60:39771
I20260808 16:52:28.683451 19658 heartbeater.cc:464] Registering TS with master...
I20260808 16:52:28.683509 19658 heartbeater.cc:511] Master 127.8.24.60:39771 requested a full tablet report, sending...
I20260808 16:52:28.683835 19150 ts_manager.cc:194] Registered new tserver with Master: 005e71fe78a847c2b010b116c89a8609 (127.8.24.2:38759)
I20260808 16:52:28.687254 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.3:40919
--local_ip_for_outbound_sockets=127.8.24.3
--tserver_master_addrs=127.8.24.62:46193
--webserver_port=35665
--webserver_interface=127.8.24.3
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/krb5kdc/oryx_127.8.24.3.keytab
--principal=oryx/127.8.24.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:37767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.8.24.62:46193,127.8.24.61:45755
--tserver_master_addrs=127.8.24.62:46193,127.8.24.61:45755,127.8.24.60:39771 with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 16:52:28.738498 19215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:28.735717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55802 (local address 127.8.24.60:39771)
0808 16:52:28.735766 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:28.735768 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:28.735777 (+ 9us) server_negotiation.cc:407] Connection header received
0808 16:52:28.735821 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:28.735821 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:28.735838 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:28.735888 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:52:28.736004 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:28.736455 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:28.737723 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:28.738106 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:28.738173 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:28.738229 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:28.738403 (+ 174us) server_negotiation.cc:299] Negotiation successful
0808 16:52:28.738434 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 16:52:28.738505 18779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:28.735646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.60:39771 (local address 127.0.0.1:55802)
0808 16:52:28.735706 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:28.735718 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:28.735767 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:28.735903 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:28.735904 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:28.735905 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:52:28.735949 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:28.735951 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:28.736515 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:28.736516 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:28.737646 (+ 1130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:28.737649 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:28.738184 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:28.738185 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:28.738252 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:28.738256 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 16:52:28.738377 (+ 121us) client_negotiation.cc:240] Negotiation successful
0808 16:52:28.738431 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260808 16:52:28.738794 19167 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 2a210ec3273f40d1b29270637e35af7b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260808 16:52:28.738929 19209 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a210ec3273f40d1b29270637e35af7b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 678b35993e9b49aca0f3ad2bba5edfbd. Latest consensus state: current_term: 1 leader_uuid: "678b35993e9b49aca0f3ad2bba5edfbd" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "678b35993e9b49aca0f3ad2bba5edfbd" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 46193 } } peers { permanent_uuid: "ca479471edff409d957dde1bb92c4144" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 45755 } attrs { promote: false } } peers { permanent_uuid: "2a210ec3273f40d1b29270637e35af7b" member_type: NON_VOTER last_known_addr { host: "127.8.24.60" port: 39771 } attrs { promote: true } } }
I20260808 16:52:28.739017 19209 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a210ec3273f40d1b29270637e35af7b [sys.catalog]: This master's current role is: LEARNER
W20260808 16:52:28.767068 19666 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:52:28.767257 19666 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:52:28.767283 19666 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:52:28.768371 19666 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:52:28.768409 19666 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:52:28.768432 19666 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.3
I20260808 16:52:28.769825 19666 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:37767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/krb5kdc/oryx_127.8.24.3.keytab
--principal=oryx/127.8.24.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.3:40919
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.24.3
--webserver_port=35665
--tserver_master_addrs=127.8.24.62:46193,127.8.24.61:45755,127.8.24.60:39771
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19666
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.3
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:52:28.770026 19666 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:52:28.770244 19666 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:52:28.774848 19680 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:28.774873 19682 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:28.774873 19677 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:28.774924 19666 server_base.cc:1034] running on GCE node
I20260808 16:52:28.775214 19666 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:52:28.775430 19666 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:52:28.776558 19666 hybrid_clock.cc:648] HybridClock initialized: now 1786207948776549 us; error 28 us; skew 500 ppm
Aug 08 16:52:28 dist-test-slave-6zbq krb5kdc[17768](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207948, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.24.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:52:28.778733 19666 init.cc:377] Logged in from keytab as oryx/127.8.24.3@KRBTEST.COM (short username oryx)
I20260808 16:52:28.779088 19666 webserver.cc:460] Webserver started at http://127.8.24.3:35665/ using document root <none> and password file <none>
I20260808 16:52:28.779273 19666 fs_manager.cc:359] Metadata directory not provided
I20260808 16:52:28.779318 19666 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:52:28.780483 19666 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 16:52:28.780970 19693 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:28.781118 19666 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:28.781204 19666 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/ts-2/wal
uuid: "5f52c81c0e744fd195de3a1f27250198"
format_stamp: "Formatted at 2026-08-08 16:52:27 on dist-test-slave-6zbq"
server_key: "4190edc4403864b54992fc64510692f8"
server_key_iv: "095d89c1b6e40fd0bb3a8944d9ba0f20"
server_key_version: "encryptionkey@0"
I20260808 16:52:28.781510 19666 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:52:28.793007 19666 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:52:28.794538 19666 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:52:28.794652 19666 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:52:28.794835 19666 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:52:28.795086 19666 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:52:28.795117 19666 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:28.795136 19666 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:52:28.795167 19666 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:28.800402 19666 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.3:40919
I20260808 16:52:28.800458 19890 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.3:40919 every 8 connection(s)
I20260808 16:52:28.800721 19666 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/ts-2/data/info.pb
I20260808 16:52:28.801822 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 19666
Aug 08 16:52:28 dist-test-slave-6zbq krb5kdc[17768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207948, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.24.3@KRBTEST.COM for oryx/127.8.24.61@KRBTEST.COM
Aug 08 16:52:28 dist-test-slave-6zbq krb5kdc[17768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207948, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.24.3@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM
Aug 08 16:52:28 dist-test-slave-6zbq krb5kdc[17768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207948, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.24.3@KRBTEST.COM for oryx/127.8.24.60@KRBTEST.COM
I20260808 16:52:28.813791 19914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:28.803493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59034 (local address 127.8.24.62:46193)
0808 16:52:28.803625 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:28.803627 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:28.803881 (+ 254us) server_negotiation.cc:407] Connection header received
0808 16:52:28.803934 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:28.803935 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:28.803947 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:28.804014 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:28.804932 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:28.805623 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:28.806648 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:28.806932 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:28.806990 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:28.808726 (+ 1736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:28.808727 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:28.808727 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:28.808734 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:28.812470 (+ 3736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:28.812858 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:28.812859 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:28.812860 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:28.812922 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:28.813137 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:28.813138 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:28.813139 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:28.813285 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:28.813361 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:28.813601 (+ 240us) server_negotiation.cc:299] Negotiation successful
0808 16:52:28.813690 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":72,"threads_started":1}
I20260808 16:52:28.813942 17917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:28.801293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:48047 (local address 127.8.24.62:46193)
0808 16:52:28.801340 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:28.801342 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:28.801610 (+ 268us) server_negotiation.cc:407] Connection header received
0808 16:52:28.802745 (+ 1135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:28.802747 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:28.802773 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:28.802801 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:28.803859 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:28.804578 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:28.806709 (+ 2131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:28.806985 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:28.807036 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:28.810248 (+ 3212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:28.810249 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:28.810250 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:28.810256 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:28.811941 (+ 1685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:28.812356 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:28.812357 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:28.812357 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:28.812417 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:28.812993 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:28.812995 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:28.812995 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:28.813124 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:28.813164 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:28.813697 (+ 533us) server_negotiation.cc:299] Negotiation successful
0808 16:52:28.813775 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 16:52:28.814877 19896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:28.801317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:46193 (local address 127.8.24.3:48047)
0808 16:52:28.801486 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:28.801524 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:28.802631 (+ 1107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:28.802961 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:28.802996 (+ 35us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:28.803483 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:28.803742 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:28.803748 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:28.805311 (+ 1563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:28.805312 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:28.806636 (+ 1324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:28.806638 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:28.807094 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:28.807095 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:28.807242 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:28.808925 (+ 1683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:28.808930 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:28.810152 (+ 1222us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:52:28.812015 (+ 1863us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:28.812022 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:28.812024 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:28.812288 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:28.812466 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:28.812466 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:28.812467 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:28.812590 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:28.813355 (+ 765us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:52:28.813362 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:52:28.813509 (+ 147us) client_negotiation.cc:769] Sending connection context
0808 16:52:28.814513 (+ 1004us) client_negotiation.cc:240] Negotiation successful
0808 16:52:28.814683 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"mutex_wait_us":4,"thread_start_us":103,"threads_started":1}
I20260808 16:52:28.815347 19893 heartbeater.cc:347] Connected to a master server at 127.8.24.62:46193
I20260808 16:52:28.815438 19893 heartbeater.cc:464] Registering TS with master...
I20260808 16:52:28.815639 19893 heartbeater.cc:511] Master 127.8.24.62:46193 requested a full tablet report, sending...
I20260808 16:52:28.816046 19215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:28.801672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:49865 (local address 127.8.24.60:39771)
0808 16:52:28.801733 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:28.801736 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:28.802485 (+ 749us) server_negotiation.cc:407] Connection header received
0808 16:52:28.802767 (+ 282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:28.802768 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:28.802780 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:28.802825 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:28.804522 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:28.805214 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:28.808294 (+ 3080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:28.808549 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:28.808589 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:28.811838 (+ 3249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:28.811839 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:28.811840 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:28.811847 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:28.813864 (+ 2017us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:28.815146 (+ 1282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:28.815147 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:28.815147 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:28.815211 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:28.815466 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:28.815467 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:28.815467 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:28.815587 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:28.815633 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:28.815893 (+ 260us) server_negotiation.cc:299] Negotiation successful
0808 16:52:28.815966 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 16:52:28.816046 17841 ts_manager.cc:194] Re-registered known tserver with Master: 5f52c81c0e744fd195de3a1f27250198 (127.8.24.3:40919)
I20260808 16:52:28.816268 19907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:28.802899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55324 (local address 127.8.24.61:45755)
0808 16:52:28.804035 (+ 1136us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:28.804037 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:28.804044 (+ 7us) server_negotiation.cc:407] Connection header received
0808 16:52:28.804060 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:28.804061 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:28.804071 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:28.804390 (+ 319us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:28.804914 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:28.805577 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:28.809817 (+ 4240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:28.810104 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:28.810178 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:28.811434 (+ 1256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:28.811435 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:28.811436 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:28.811442 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:28.814586 (+ 3144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:28.814918 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:28.814919 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:28.814920 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:28.815008 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:28.815342 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:28.815343 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:28.815344 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:28.815727 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:28.816066 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:28.816107 (+ 41us) server_negotiation.cc:299] Negotiation successful
0808 16:52:28.816181 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":477,"thread_start_us":54,"threads_started":1}
I20260808 16:52:28.816501 17841 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.24.3@KRBTEST.COM'} at 127.8.24.3:48047
I20260808 16:52:28.816679 19898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:28.801588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.60:39771 (local address 127.8.24.3:49865)
0808 16:52:28.802415 (+ 827us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:28.802425 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:28.802645 (+ 220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:28.802962 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:28.802966 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:28.803903 (+ 937us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:28.803965 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:28.803968 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:28.807443 (+ 3475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:28.807443 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:28.808192 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:28.808194 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:28.808681 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:28.808682 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:28.808748 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:28.810514 (+ 1766us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:28.810518 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:28.811761 (+ 1243us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:52:28.814386 (+ 2625us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:28.814387 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:28.814388 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:28.814581 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:28.815282 (+ 701us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:28.815282 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:28.815283 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:28.815341 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:28.815741 (+ 400us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:52:28.815742 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:52:28.815783 (+ 41us) client_negotiation.cc:769] Sending connection context
0808 16:52:28.816287 (+ 504us) client_negotiation.cc:240] Negotiation successful
0808 16:52:28.816535 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":810,"thread_start_us":58,"threads_started":1}
I20260808 16:52:28.817194 19900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:28.801317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.61:45755 (local address 127.8.24.3:45753)
0808 16:52:28.801693 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:28.801703 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:28.802632 (+ 929us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:28.802961 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:28.802965 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:28.803446 (+ 481us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:28.803742 (+ 296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:28.803748 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:28.805440 (+ 1692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:28.805441 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:28.806747 (+ 1306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:28.806748 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:28.807192 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:28.807193 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:28.807246 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:28.807784 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:28.807793 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:28.809426 (+ 1633us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:52:28.814662 (+ 5236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:28.814663 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:28.814663 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:28.814853 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:28.815440 (+ 587us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:28.815441 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:28.815441 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:28.816586 (+ 1145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:28.816895 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:52:28.816896 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:52:28.816934 (+ 38us) client_negotiation.cc:769] Sending connection context
0808 16:52:28.816992 (+ 58us) client_negotiation.cc:240] Negotiation successful
0808 16:52:28.817080 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":347,"mutex_wait_us":2,"thread_start_us":89,"threads_started":1}
I20260808 16:52:28.817210 19916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:28.803696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55816 (local address 127.8.24.60:39771)
0808 16:52:28.803798 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:28.803800 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:28.804045 (+ 245us) server_negotiation.cc:407] Connection header received
0808 16:52:28.804071 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:28.804071 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:28.804084 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:28.804134 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:28.806027 (+ 1893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:28.806690 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:28.808561 (+ 1871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:28.808834 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:28.808890 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:28.811983 (+ 3093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:28.811984 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:28.811985 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:28.811991 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:28.814429 (+ 2438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:28.815850 (+ 1421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:28.815851 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:28.815851 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:28.815909 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:28.816186 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:28.816188 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:28.816188 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:28.816308 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:28.816683 (+ 375us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:28.817037 (+ 354us) server_negotiation.cc:299] Negotiation successful
0808 16:52:28.817107 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85,"thread_start_us":56,"threads_started":1}
I20260808 16:52:28.817248 18924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:28.801339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:45753 (local address 127.8.24.61:45755)
0808 16:52:28.801378 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:28.801379 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:28.801753 (+ 374us) server_negotiation.cc:407] Connection header received
0808 16:52:28.802747 (+ 994us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:28.802748 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:28.802759 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:28.802790 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:28.804675 (+ 1885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:28.805358 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:28.806816 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:28.807086 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:28.807123 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:28.809529 (+ 2406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:28.809530 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:28.809530 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:28.809537 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:28.813039 (+ 3502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:28.815159 (+ 2120us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:28.815161 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:28.815161 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:28.815235 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:28.816684 (+ 1449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:28.816685 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:28.816685 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:28.816815 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:28.816863 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:28.817096 (+ 233us) server_negotiation.cc:299] Negotiation successful
0808 16:52:28.817175 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 16:52:28.817759 19894 heartbeater.cc:347] Connected to a master server at 127.8.24.61:45755
I20260808 16:52:28.817806 19894 heartbeater.cc:464] Registering TS with master...
I20260808 16:52:28.817869 19894 heartbeater.cc:511] Master 127.8.24.61:45755 requested a full tablet report, sending...
I20260808 16:52:28.818120 18847 ts_manager.cc:194] Registered new tserver with Master: 5f52c81c0e744fd195de3a1f27250198 (127.8.24.3:40919)
I20260808 16:52:28.818239 19891 heartbeater.cc:347] Connected to a master server at 127.8.24.60:39771
I20260808 16:52:28.818295 19891 heartbeater.cc:464] Registering TS with master...
I20260808 16:52:28.818356 19891 heartbeater.cc:511] Master 127.8.24.60:39771 requested a full tablet report, sending...
I20260808 16:52:28.818596 19150 ts_manager.cc:194] Registered new tserver with Master: 5f52c81c0e744fd195de3a1f27250198 (127.8.24.3:40919)
I20260808 16:52:28.816514 17840 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59034:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 16:52:28.818897 17840 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 16:52:28.819512 18780 sys_catalog.cc:455] T 00000000000000000000000000000000 P 678b35993e9b49aca0f3ad2bba5edfbd [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "678b35993e9b49aca0f3ad2bba5edfbd" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "678b35993e9b49aca0f3ad2bba5edfbd" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 46193 } } peers { permanent_uuid: "ca479471edff409d957dde1bb92c4144" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 45755 } attrs { promote: false } } peers { permanent_uuid: "2a210ec3273f40d1b29270637e35af7b" member_type: NON_VOTER last_known_addr { host: "127.8.24.60" port: 39771 } attrs { promote: true } } }
I20260808 16:52:28.819600 18780 sys_catalog.cc:458] T 00000000000000000000000000000000 P 678b35993e9b49aca0f3ad2bba5edfbd [sys.catalog]: This master's current role is: LEADER
I20260808 16:52:28.821115 18780 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 678b35993e9b49aca0f3ad2bba5edfbd: attempting to promote NON_VOTER 2a210ec3273f40d1b29270637e35af7b to VOTER
I20260808 16:52:28.821234 18780 consensus_queue.cc:237] T 00000000000000000000000000000000 P 678b35993e9b49aca0f3ad2bba5edfbd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "678b35993e9b49aca0f3ad2bba5edfbd" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 46193 } } peers { permanent_uuid: "ca479471edff409d957dde1bb92c4144" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 45755 } attrs { promote: false } } peers { permanent_uuid: "2a210ec3273f40d1b29270637e35af7b" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 39771 } attrs { promote: false } }
I20260808 16:52:28.821846 19165 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2a210ec3273f40d1b29270637e35af7b [term 1 LEARNER]: Refusing update from remote peer 678b35993e9b49aca0f3ad2bba5edfbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260808 16:52:28.822084 18860 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ca479471edff409d957dde1bb92c4144 [term 1 FOLLOWER]: Refusing update from remote peer 678b35993e9b49aca0f3ad2bba5edfbd: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20260808 16:52:28.822058 19927 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 678b35993e9b49aca0f3ad2bba5edfbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a210ec3273f40d1b29270637e35af7b" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 39771 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260808 16:52:28.822252 19927 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 678b35993e9b49aca0f3ad2bba5edfbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca479471edff409d957dde1bb92c4144" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 45755 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260808 16:52:28.822784 19921 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 678b35993e9b49aca0f3ad2bba5edfbd [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 2a210ec3273f40d1b29270637e35af7b (127.8.24.60) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "678b35993e9b49aca0f3ad2bba5edfbd" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 46193 } } peers { permanent_uuid: "ca479471edff409d957dde1bb92c4144" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 45755 } attrs { promote: false } } peers { permanent_uuid: "2a210ec3273f40d1b29270637e35af7b" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 39771 } attrs { promote: false } } }
I20260808 16:52:28.822927 19165 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 2a210ec3273f40d1b29270637e35af7b [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 2a210ec3273f40d1b29270637e35af7b (127.8.24.60) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "678b35993e9b49aca0f3ad2bba5edfbd" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 46193 } } peers { permanent_uuid: "ca479471edff409d957dde1bb92c4144" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 45755 } attrs { promote: false } } peers { permanent_uuid: "2a210ec3273f40d1b29270637e35af7b" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 39771 } attrs { promote: false } } }
I20260808 16:52:28.823573 19927 sys_catalog.cc:455] T 00000000000000000000000000000000 P 678b35993e9b49aca0f3ad2bba5edfbd [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "678b35993e9b49aca0f3ad2bba5edfbd" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "678b35993e9b49aca0f3ad2bba5edfbd" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 46193 } } peers { permanent_uuid: "ca479471edff409d957dde1bb92c4144" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 45755 } attrs { promote: false } } peers { permanent_uuid: "2a210ec3273f40d1b29270637e35af7b" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 39771 } attrs { promote: false } } }
I20260808 16:52:28.823661 19927 sys_catalog.cc:458] T 00000000000000000000000000000000 P 678b35993e9b49aca0f3ad2bba5edfbd [sys.catalog]: This master's current role is: LEADER
I20260808 16:52:28.823681 18860 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ca479471edff409d957dde1bb92c4144 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 2a210ec3273f40d1b29270637e35af7b (127.8.24.60) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "678b35993e9b49aca0f3ad2bba5edfbd" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 46193 } } peers { permanent_uuid: "ca479471edff409d957dde1bb92c4144" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 45755 } attrs { promote: false } } peers { permanent_uuid: "2a210ec3273f40d1b29270637e35af7b" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 39771 } attrs { promote: false } } }
I20260808 16:52:28.823731 19927 sys_catalog.cc:455] T 00000000000000000000000000000000 P 678b35993e9b49aca0f3ad2bba5edfbd [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "678b35993e9b49aca0f3ad2bba5edfbd" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "678b35993e9b49aca0f3ad2bba5edfbd" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 46193 } } peers { permanent_uuid: "ca479471edff409d957dde1bb92c4144" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 45755 } attrs { promote: false } } peers { permanent_uuid: "2a210ec3273f40d1b29270637e35af7b" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 39771 } attrs { promote: false } } }
I20260808 16:52:28.823773 19927 sys_catalog.cc:458] T 00000000000000000000000000000000 P 678b35993e9b49aca0f3ad2bba5edfbd [sys.catalog]: This master's current role is: LEADER
I20260808 16:52:28.823875 19209 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a210ec3273f40d1b29270637e35af7b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "678b35993e9b49aca0f3ad2bba5edfbd" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "678b35993e9b49aca0f3ad2bba5edfbd" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 46193 } } peers { permanent_uuid: "ca479471edff409d957dde1bb92c4144" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 45755 } attrs { promote: false } } peers { permanent_uuid: "2a210ec3273f40d1b29270637e35af7b" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 39771 } attrs { promote: false } } }
I20260808 16:52:28.823941 19209 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a210ec3273f40d1b29270637e35af7b [sys.catalog]: This master's current role is: FOLLOWER
I20260808 16:52:28.823587 19921 sys_catalog.cc:455] T 00000000000000000000000000000000 P 678b35993e9b49aca0f3ad2bba5edfbd [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "678b35993e9b49aca0f3ad2bba5edfbd" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "678b35993e9b49aca0f3ad2bba5edfbd" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 46193 } } peers { permanent_uuid: "ca479471edff409d957dde1bb92c4144" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 45755 } attrs { promote: false } } peers { permanent_uuid: "2a210ec3273f40d1b29270637e35af7b" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 39771 } attrs { promote: false } } }
I20260808 16:52:28.823984 19921 sys_catalog.cc:458] T 00000000000000000000000000000000 P 678b35993e9b49aca0f3ad2bba5edfbd [sys.catalog]: This master's current role is: LEADER
I20260808 16:52:28.824502 19945 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca479471edff409d957dde1bb92c4144 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "678b35993e9b49aca0f3ad2bba5edfbd" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "678b35993e9b49aca0f3ad2bba5edfbd" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 46193 } } peers { permanent_uuid: "ca479471edff409d957dde1bb92c4144" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 45755 } attrs { promote: false } } peers { permanent_uuid: "2a210ec3273f40d1b29270637e35af7b" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 39771 } attrs { promote: false } } }
I20260808 16:52:28.824661 19945 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca479471edff409d957dde1bb92c4144 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 16:52:28.828675 18779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:28.824955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.3:40919 (local address 127.0.0.1:57440)
0808 16:52:28.825175 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:28.825185 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:28.825224 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:28.825674 (+ 450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:28.825675 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:28.825677 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:52:28.825714 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:28.825715 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:28.826645 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:28.826646 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:28.827614 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:28.827616 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:28.828445 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:28.828446 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:28.828505 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:28.828509 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 16:52:28.828569 (+ 60us) client_negotiation.cc:240] Negotiation successful
0808 16:52:28.828604 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"mutex_wait_us":1}
I20260808 16:52:28.828696 19946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:28.825024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57440 (local address 127.8.24.3:40919)
0808 16:52:28.825540 (+ 516us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:28.825544 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:28.825552 (+ 8us) server_negotiation.cc:407] Connection header received
0808 16:52:28.825580 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:28.825582 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:28.825594 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:28.825659 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:52:28.825772 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:28.826579 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:28.827706 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:28.828357 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:28.828433 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:28.828499 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:28.828584 (+ 85us) server_negotiation.cc:299] Negotiation successful
0808 16:52:28.828616 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":485,"thread_start_us":88,"threads_started":1}
I20260808 16:52:28.828928 19952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:28.825029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:38887 (local address 127.0.0.1:39848)
0808 16:52:28.825289 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:28.825296 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:28.825327 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:28.825606 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:28.825606 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:28.825608 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:52:28.825653 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:28.825655 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:28.826957 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:28.826958 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:28.827949 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:28.827952 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:28.828729 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:28.828730 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:28.828791 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:28.828795 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 16:52:28.828825 (+ 30us) client_negotiation.cc:240] Negotiation successful
0808 16:52:28.828859 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"mutex_wait_us":11,"thread_start_us":97,"threads_started":1}
I20260808 16:52:28.829099 19951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:28.825035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.2:38759 (local address 127.0.0.1:45244)
0808 16:52:28.825388 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:28.825395 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:28.825428 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:28.825908 (+ 480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:28.825909 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:28.825911 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:52:28.825952 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:28.825953 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:28.827107 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:28.827108 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:28.827912 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:28.827915 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:28.828886 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:28.828887 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:28.828944 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:28.828948 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 16:52:28.828994 (+ 46us) client_negotiation.cc:240] Negotiation successful
0808 16:52:28.829033 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":96,"threads_started":1}
I20260808 16:52:28.829265 19954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:28.825270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39848 (local address 127.8.24.1:38887)
0808 16:52:28.825435 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:28.825439 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:28.825450 (+ 11us) server_negotiation.cc:407] Connection header received
0808 16:52:28.825489 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:28.825491 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:28.825508 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:28.825723 (+ 215us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:52:28.825782 (+ 59us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:28.826875 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:28.828035 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:28.828640 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:28.829011 (+ 371us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:28.829103 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:28.829129 (+ 26us) server_negotiation.cc:299] Negotiation successful
0808 16:52:28.829186 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":66,"threads_started":1}
I20260808 16:52:28.829521 19783 tablet_service.cc:1467] Processing CreateTablet for tablet a59d1aa03092471ab0ee5ed7598355d6 (DEFAULT_TABLE table=test-table [id=a86bbe410c8b4b5d81fcb3795da36291]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:52:28.829857 19783 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a59d1aa03092471ab0ee5ed7598355d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:52:28.829999 19321 tablet_service.cc:1467] Processing CreateTablet for tablet a59d1aa03092471ab0ee5ed7598355d6 (DEFAULT_TABLE table=test-table [id=a86bbe410c8b4b5d81fcb3795da36291]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:52:28.830366 19321 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a59d1aa03092471ab0ee5ed7598355d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:52:28.830519 19953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:28.825112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45244 (local address 127.8.24.2:38759)
0808 16:52:28.825774 (+ 662us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:28.825778 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:28.825787 (+ 9us) server_negotiation.cc:407] Connection header received
0808 16:52:28.825822 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:28.825824 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:28.825838 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:28.825912 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:52:28.826016 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:28.827038 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:28.828006 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:28.828715 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:28.828787 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:28.828856 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:28.830385 (+ 1529us) server_negotiation.cc:299] Negotiation successful
0808 16:52:28.830435 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":627,"thread_start_us":94,"threads_started":1}
I20260808 16:52:28.833245 19956 tablet_bootstrap.cc:492] T a59d1aa03092471ab0ee5ed7598355d6 P 5f52c81c0e744fd195de3a1f27250198: Bootstrap starting.
I20260808 16:52:28.833586 19959 tablet_bootstrap.cc:492] T a59d1aa03092471ab0ee5ed7598355d6 P 7b46262135f64e30b03b7b7cfa5e7044: Bootstrap starting.
I20260808 16:52:28.833943 19956 tablet_bootstrap.cc:654] T a59d1aa03092471ab0ee5ed7598355d6 P 5f52c81c0e744fd195de3a1f27250198: Neither blocks nor log segments found. Creating new log.
I20260808 16:52:28.834148 19959 tablet_bootstrap.cc:654] T a59d1aa03092471ab0ee5ed7598355d6 P 7b46262135f64e30b03b7b7cfa5e7044: Neither blocks nor log segments found. Creating new log.
I20260808 16:52:28.834267 19956 log.cc:824] T a59d1aa03092471ab0ee5ed7598355d6 P 5f52c81c0e744fd195de3a1f27250198: Log is configured to *not* fsync() on all Append() calls
I20260808 16:52:28.834194 19545 tablet_service.cc:1467] Processing CreateTablet for tablet a59d1aa03092471ab0ee5ed7598355d6 (DEFAULT_TABLE table=test-table [id=a86bbe410c8b4b5d81fcb3795da36291]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:52:28.834481 19959 log.cc:824] T a59d1aa03092471ab0ee5ed7598355d6 P 7b46262135f64e30b03b7b7cfa5e7044: Log is configured to *not* fsync() on all Append() calls
I20260808 16:52:28.834551 19545 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a59d1aa03092471ab0ee5ed7598355d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:52:28.835070 19956 tablet_bootstrap.cc:492] T a59d1aa03092471ab0ee5ed7598355d6 P 5f52c81c0e744fd195de3a1f27250198: No bootstrap required, opened a new log
I20260808 16:52:28.835146 19956 ts_tablet_manager.cc:1397] T a59d1aa03092471ab0ee5ed7598355d6 P 5f52c81c0e744fd195de3a1f27250198: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 16:52:28.835332 19959 tablet_bootstrap.cc:492] T a59d1aa03092471ab0ee5ed7598355d6 P 7b46262135f64e30b03b7b7cfa5e7044: No bootstrap required, opened a new log
I20260808 16:52:28.835443 19959 ts_tablet_manager.cc:1397] T a59d1aa03092471ab0ee5ed7598355d6 P 7b46262135f64e30b03b7b7cfa5e7044: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 16:52:28.836920 19966 tablet_bootstrap.cc:492] T a59d1aa03092471ab0ee5ed7598355d6 P 005e71fe78a847c2b010b116c89a8609: Bootstrap starting.
I20260808 16:52:28.837570 19966 tablet_bootstrap.cc:654] T a59d1aa03092471ab0ee5ed7598355d6 P 005e71fe78a847c2b010b116c89a8609: Neither blocks nor log segments found. Creating new log.
I20260808 16:52:28.837495 19956 raft_consensus.cc:348] T a59d1aa03092471ab0ee5ed7598355d6 P 5f52c81c0e744fd195de3a1f27250198 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f52c81c0e744fd195de3a1f27250198" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40919 } } peers { permanent_uuid: "7b46262135f64e30b03b7b7cfa5e7044" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 38887 } } peers { permanent_uuid: "005e71fe78a847c2b010b116c89a8609" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 38759 } }
I20260808 16:52:28.837653 19956 raft_consensus.cc:374] T a59d1aa03092471ab0ee5ed7598355d6 P 5f52c81c0e744fd195de3a1f27250198 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:52:28.837705 19956 raft_consensus.cc:729] T a59d1aa03092471ab0ee5ed7598355d6 P 5f52c81c0e744fd195de3a1f27250198 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f52c81c0e744fd195de3a1f27250198, State: Initialized, Role: FOLLOWER
I20260808 16:52:28.837821 19956 consensus_queue.cc:260] T a59d1aa03092471ab0ee5ed7598355d6 P 5f52c81c0e744fd195de3a1f27250198 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f52c81c0e744fd195de3a1f27250198" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40919 } } peers { permanent_uuid: "7b46262135f64e30b03b7b7cfa5e7044" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 38887 } } peers { permanent_uuid: "005e71fe78a847c2b010b116c89a8609" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 38759 } }
I20260808 16:52:28.837877 19966 log.cc:824] T a59d1aa03092471ab0ee5ed7598355d6 P 005e71fe78a847c2b010b116c89a8609: Log is configured to *not* fsync() on all Append() calls
I20260808 16:52:28.838068 19956 ts_tablet_manager.cc:1428] T a59d1aa03092471ab0ee5ed7598355d6 P 5f52c81c0e744fd195de3a1f27250198: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260808 16:52:28.838268 19959 raft_consensus.cc:348] T a59d1aa03092471ab0ee5ed7598355d6 P 7b46262135f64e30b03b7b7cfa5e7044 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f52c81c0e744fd195de3a1f27250198" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40919 } } peers { permanent_uuid: "7b46262135f64e30b03b7b7cfa5e7044" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 38887 } } peers { permanent_uuid: "005e71fe78a847c2b010b116c89a8609" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 38759 } }
I20260808 16:52:28.838439 19893 heartbeater.cc:503] Master 127.8.24.62:46193 was elected leader, sending a full tablet report...
I20260808 16:52:28.838438 19959 raft_consensus.cc:374] T a59d1aa03092471ab0ee5ed7598355d6 P 7b46262135f64e30b03b7b7cfa5e7044 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:52:28.838479 19959 raft_consensus.cc:729] T a59d1aa03092471ab0ee5ed7598355d6 P 7b46262135f64e30b03b7b7cfa5e7044 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b46262135f64e30b03b7b7cfa5e7044, State: Initialized, Role: FOLLOWER
I20260808 16:52:28.838577 19959 consensus_queue.cc:260] T a59d1aa03092471ab0ee5ed7598355d6 P 7b46262135f64e30b03b7b7cfa5e7044 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f52c81c0e744fd195de3a1f27250198" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40919 } } peers { permanent_uuid: "7b46262135f64e30b03b7b7cfa5e7044" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 38887 } } peers { permanent_uuid: "005e71fe78a847c2b010b116c89a8609" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 38759 } }
I20260808 16:52:28.838817 19959 ts_tablet_manager.cc:1428] T a59d1aa03092471ab0ee5ed7598355d6 P 7b46262135f64e30b03b7b7cfa5e7044: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 16:52:28.838850 19440 heartbeater.cc:503] Master 127.8.24.62:46193 was elected leader, sending a full tablet report...
I20260808 16:52:28.839429 19966 tablet_bootstrap.cc:492] T a59d1aa03092471ab0ee5ed7598355d6 P 005e71fe78a847c2b010b116c89a8609: No bootstrap required, opened a new log
I20260808 16:52:28.839548 19966 ts_tablet_manager.cc:1397] T a59d1aa03092471ab0ee5ed7598355d6 P 005e71fe78a847c2b010b116c89a8609: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260808 16:52:28.841694 19966 raft_consensus.cc:348] T a59d1aa03092471ab0ee5ed7598355d6 P 005e71fe78a847c2b010b116c89a8609 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f52c81c0e744fd195de3a1f27250198" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40919 } } peers { permanent_uuid: "7b46262135f64e30b03b7b7cfa5e7044" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 38887 } } peers { permanent_uuid: "005e71fe78a847c2b010b116c89a8609" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 38759 } }
I20260808 16:52:28.841814 19966 raft_consensus.cc:374] T a59d1aa03092471ab0ee5ed7598355d6 P 005e71fe78a847c2b010b116c89a8609 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:52:28.841833 19966 raft_consensus.cc:729] T a59d1aa03092471ab0ee5ed7598355d6 P 005e71fe78a847c2b010b116c89a8609 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 005e71fe78a847c2b010b116c89a8609, State: Initialized, Role: FOLLOWER
I20260808 16:52:28.841935 19966 consensus_queue.cc:260] T a59d1aa03092471ab0ee5ed7598355d6 P 005e71fe78a847c2b010b116c89a8609 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f52c81c0e744fd195de3a1f27250198" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40919 } } peers { permanent_uuid: "7b46262135f64e30b03b7b7cfa5e7044" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 38887 } } peers { permanent_uuid: "005e71fe78a847c2b010b116c89a8609" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 38759 } }
I20260808 16:52:28.842146 19966 ts_tablet_manager.cc:1428] T a59d1aa03092471ab0ee5ed7598355d6 P 005e71fe78a847c2b010b116c89a8609: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260808 16:52:28.842183 19659 heartbeater.cc:503] Master 127.8.24.62:46193 was elected leader, sending a full tablet report...
W20260808 16:52:28.900339 19662 tablet.cc:2369] T a59d1aa03092471ab0ee5ed7598355d6 P 005e71fe78a847c2b010b116c89a8609: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 16:52:28.988123 19442 tablet.cc:2369] T a59d1aa03092471ab0ee5ed7598355d6 P 7b46262135f64e30b03b7b7cfa5e7044: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 16:52:29.027153 19970 raft_consensus.cc:482] T a59d1aa03092471ab0ee5ed7598355d6 P 7b46262135f64e30b03b7b7cfa5e7044 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 16:52:29.027240 19970 raft_consensus.cc:504] T a59d1aa03092471ab0ee5ed7598355d6 P 7b46262135f64e30b03b7b7cfa5e7044 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f52c81c0e744fd195de3a1f27250198" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40919 } } peers { permanent_uuid: "7b46262135f64e30b03b7b7cfa5e7044" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 38887 } } peers { permanent_uuid: "005e71fe78a847c2b010b116c89a8609" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 38759 } }
I20260808 16:52:29.027546 19970 leader_election.cc:288] T a59d1aa03092471ab0ee5ed7598355d6 P 7b46262135f64e30b03b7b7cfa5e7044 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5f52c81c0e744fd195de3a1f27250198 (127.8.24.3:40919), 005e71fe78a847c2b010b116c89a8609 (127.8.24.2:38759)
I20260808 16:52:29.030190 19953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:29.027797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:44747 (local address 127.8.24.2:38759)
0808 16:52:29.027846 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:29.027848 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:29.027880 (+ 32us) server_negotiation.cc:407] Connection header received
0808 16:52:29.027956 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:29.027957 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:29.027966 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:29.028010 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:52:29.028128 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:29.028583 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:29.029517 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:29.029893 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:29.029953 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:29.029981 (+ 28us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:29.030088 (+ 107us) server_negotiation.cc:299] Negotiation successful
0808 16:52:29.030128 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 16:52:29.030215 19946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:29.027735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:47373 (local address 127.8.24.3:40919)
0808 16:52:29.027782 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:29.027784 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:29.027895 (+ 111us) server_negotiation.cc:407] Connection header received
0808 16:52:29.027947 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:29.027948 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:29.027959 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:29.028008 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:52:29.028137 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:29.028619 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:29.029507 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:29.029919 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:29.029976 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:29.029986 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:29.030111 (+ 125us) server_negotiation.cc:299] Negotiation successful
0808 16:52:29.030143 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 16:52:29.030233 19972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:29.027702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.2:38759 (local address 127.8.24.1:44747)
0808 16:52:29.027830 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:29.027839 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:29.027893 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:29.028019 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:29.028020 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:29.028022 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:52:29.028074 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:29.028076 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:29.028653 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:29.028654 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:29.029434 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:29.029437 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:29.029962 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:29.029963 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:29.030021 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:29.030025 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 16:52:29.030073 (+ 48us) client_negotiation.cc:240] Negotiation successful
0808 16:52:29.030114 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"mutex_wait_us":41,"thread_start_us":62,"threads_started":1}
I20260808 16:52:29.030248 19973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:29.027696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.3:40919 (local address 127.8.24.1:47373)
0808 16:52:29.027825 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:29.027835 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:29.027894 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:29.028016 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:29.028017 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:29.028019 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:52:29.028084 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:29.028085 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:29.028691 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:29.028692 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:29.029428 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:29.029430 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:29.029986 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:29.029988 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:29.030051 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:29.030055 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 16:52:29.030100 (+ 45us) client_negotiation.cc:240] Negotiation successful
0808 16:52:29.030139 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"mutex_wait_us":2,"thread_start_us":61,"threads_started":1}
I20260808 16:52:29.030568 19573 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a59d1aa03092471ab0ee5ed7598355d6" candidate_uuid: "7b46262135f64e30b03b7b7cfa5e7044" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "005e71fe78a847c2b010b116c89a8609" is_pre_election: true
I20260808 16:52:29.030716 19573 raft_consensus.cc:2454] T a59d1aa03092471ab0ee5ed7598355d6 P 005e71fe78a847c2b010b116c89a8609 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7b46262135f64e30b03b7b7cfa5e7044 in term 0.
I20260808 16:52:29.030869 19818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a59d1aa03092471ab0ee5ed7598355d6" candidate_uuid: "7b46262135f64e30b03b7b7cfa5e7044" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f52c81c0e744fd195de3a1f27250198" is_pre_election: true
I20260808 16:52:29.030931 19263 leader_election.cc:302] T a59d1aa03092471ab0ee5ed7598355d6 P 7b46262135f64e30b03b7b7cfa5e7044 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 005e71fe78a847c2b010b116c89a8609, 7b46262135f64e30b03b7b7cfa5e7044; no voters:
I20260808 16:52:29.031008 19818 raft_consensus.cc:2454] T a59d1aa03092471ab0ee5ed7598355d6 P 5f52c81c0e744fd195de3a1f27250198 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7b46262135f64e30b03b7b7cfa5e7044 in term 0.
I20260808 16:52:29.031030 19970 raft_consensus.cc:2781] T a59d1aa03092471ab0ee5ed7598355d6 P 7b46262135f64e30b03b7b7cfa5e7044 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 16:52:29.031082 19970 raft_consensus.cc:482] T a59d1aa03092471ab0ee5ed7598355d6 P 7b46262135f64e30b03b7b7cfa5e7044 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 16:52:29.031111 19970 raft_consensus.cc:3037] T a59d1aa03092471ab0ee5ed7598355d6 P 7b46262135f64e30b03b7b7cfa5e7044 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:52:29.031826 19970 raft_consensus.cc:504] T a59d1aa03092471ab0ee5ed7598355d6 P 7b46262135f64e30b03b7b7cfa5e7044 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f52c81c0e744fd195de3a1f27250198" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40919 } } peers { permanent_uuid: "7b46262135f64e30b03b7b7cfa5e7044" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 38887 } } peers { permanent_uuid: "005e71fe78a847c2b010b116c89a8609" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 38759 } }
I20260808 16:52:29.032022 19970 leader_election.cc:288] T a59d1aa03092471ab0ee5ed7598355d6 P 7b46262135f64e30b03b7b7cfa5e7044 [CANDIDATE]: Term 1 election: Requested vote from peers 5f52c81c0e744fd195de3a1f27250198 (127.8.24.3:40919), 005e71fe78a847c2b010b116c89a8609 (127.8.24.2:38759)
I20260808 16:52:29.032058 19818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a59d1aa03092471ab0ee5ed7598355d6" candidate_uuid: "7b46262135f64e30b03b7b7cfa5e7044" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f52c81c0e744fd195de3a1f27250198"
I20260808 16:52:29.032101 19573 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a59d1aa03092471ab0ee5ed7598355d6" candidate_uuid: "7b46262135f64e30b03b7b7cfa5e7044" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "005e71fe78a847c2b010b116c89a8609"
I20260808 16:52:29.032138 19818 raft_consensus.cc:3037] T a59d1aa03092471ab0ee5ed7598355d6 P 5f52c81c0e744fd195de3a1f27250198 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:52:29.032177 19573 raft_consensus.cc:3037] T a59d1aa03092471ab0ee5ed7598355d6 P 005e71fe78a847c2b010b116c89a8609 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:52:29.032842 19573 raft_consensus.cc:2454] T a59d1aa03092471ab0ee5ed7598355d6 P 005e71fe78a847c2b010b116c89a8609 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7b46262135f64e30b03b7b7cfa5e7044 in term 1.
I20260808 16:52:29.032928 19818 raft_consensus.cc:2454] T a59d1aa03092471ab0ee5ed7598355d6 P 5f52c81c0e744fd195de3a1f27250198 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7b46262135f64e30b03b7b7cfa5e7044 in term 1.
I20260808 16:52:29.032991 19263 leader_election.cc:302] T a59d1aa03092471ab0ee5ed7598355d6 P 7b46262135f64e30b03b7b7cfa5e7044 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 005e71fe78a847c2b010b116c89a8609, 7b46262135f64e30b03b7b7cfa5e7044; no voters:
I20260808 16:52:29.033072 19970 raft_consensus.cc:2781] T a59d1aa03092471ab0ee5ed7598355d6 P 7b46262135f64e30b03b7b7cfa5e7044 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:52:29.033178 19970 raft_consensus.cc:686] T a59d1aa03092471ab0ee5ed7598355d6 P 7b46262135f64e30b03b7b7cfa5e7044 [term 1 LEADER]: Becoming Leader. State: Replica: 7b46262135f64e30b03b7b7cfa5e7044, State: Running, Role: LEADER
I20260808 16:52:29.033325 19970 consensus_queue.cc:237] T a59d1aa03092471ab0ee5ed7598355d6 P 7b46262135f64e30b03b7b7cfa5e7044 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f52c81c0e744fd195de3a1f27250198" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40919 } } peers { permanent_uuid: "7b46262135f64e30b03b7b7cfa5e7044" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 38887 } } peers { permanent_uuid: "005e71fe78a847c2b010b116c89a8609" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 38759 } }
I20260808 16:52:29.034018 17840 catalog_manager.cc:5495] T a59d1aa03092471ab0ee5ed7598355d6 P 7b46262135f64e30b03b7b7cfa5e7044 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7b46262135f64e30b03b7b7cfa5e7044 (127.8.24.1). New cstate: current_term: 1 leader_uuid: "7b46262135f64e30b03b7b7cfa5e7044" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f52c81c0e744fd195de3a1f27250198" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7b46262135f64e30b03b7b7cfa5e7044" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 38887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "005e71fe78a847c2b010b116c89a8609" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 38759 } health_report { overall_health: UNKNOWN } } }
W20260808 16:52:29.051316 19895 tablet.cc:2369] T a59d1aa03092471ab0ee5ed7598355d6 P 5f52c81c0e744fd195de3a1f27250198: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 16:52:29.052292 19954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:29.050037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39862 (local address 127.8.24.1:38887)
0808 16:52:29.050078 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:29.050079 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:29.050088 (+ 9us) server_negotiation.cc:407] Connection header received
0808 16:52:29.050136 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:29.050136 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:29.050154 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:29.050210 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 16:52:29.050321 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:29.050825 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:29.051643 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:29.051825 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:29.051895 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:29.052018 (+ 123us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 16:52:29.052102 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 16:52:29.052167 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:29.052216 (+ 49us) server_negotiation.cc:299] Negotiation successful
0808 16:52:29.052235 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 16:52:29.053673 19573 raft_consensus.cc:1261] T a59d1aa03092471ab0ee5ed7598355d6 P 005e71fe78a847c2b010b116c89a8609 [term 1 FOLLOWER]: Refusing update from remote peer 7b46262135f64e30b03b7b7cfa5e7044: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:52:29.053699 19818 raft_consensus.cc:1261] T a59d1aa03092471ab0ee5ed7598355d6 P 5f52c81c0e744fd195de3a1f27250198 [term 1 FOLLOWER]: Refusing update from remote peer 7b46262135f64e30b03b7b7cfa5e7044: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:52:29.053957 19975 consensus_queue.cc:1035] T a59d1aa03092471ab0ee5ed7598355d6 P 7b46262135f64e30b03b7b7cfa5e7044 [LEADER]: Connected to new peer: Peer: permanent_uuid: "005e71fe78a847c2b010b116c89a8609" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 38759 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:52:29.054071 19970 consensus_queue.cc:1035] T a59d1aa03092471ab0ee5ed7598355d6 P 7b46262135f64e30b03b7b7cfa5e7044 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f52c81c0e744fd195de3a1f27250198" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:52:29.056254 19981 mvcc.cc:205] Tried to move back new op lower bound from 7316307759321907200 to 7316307759241097216. Current Snapshot: MvccSnapshot[applied={T|T < 7316307759321907200}]
I20260808 16:52:29.057410 19982 mvcc.cc:205] Tried to move back new op lower bound from 7316307759321907200 to 7316307759241097216. Current Snapshot: MvccSnapshot[applied={T|T < 7316307759321907200 or (T in {7316307759321907200})}]
I20260808 16:52:29.058825 19983 mvcc.cc:205] Tried to move back new op lower bound from 7316307759321907200 to 7316307759241097216. Current Snapshot: MvccSnapshot[applied={T|T < 7316307759321907200 or (T in {7316307759321907200})}]
I20260808 16:52:29.059532 17841 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59034:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 16:52:29.059607 17841 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59034:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 16:52:29.060797 17841 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 678b35993e9b49aca0f3ad2bba5edfbd: Sending DeleteTablet for 3 replicas of tablet a59d1aa03092471ab0ee5ed7598355d6
I20260808 16:52:29.061273 19783 tablet_service.cc:1514] Processing DeleteTablet for tablet a59d1aa03092471ab0ee5ed7598355d6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:52:29 UTC) from {username='oryx', principal='oryx/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:57440
I20260808 16:52:29.061396 19545 tablet_service.cc:1514] Processing DeleteTablet for tablet a59d1aa03092471ab0ee5ed7598355d6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:52:29 UTC) from {username='oryx', principal='oryx/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:45244
I20260808 16:52:29.061499 19994 tablet_replica.cc:330] T a59d1aa03092471ab0ee5ed7598355d6 P 5f52c81c0e744fd195de3a1f27250198: stopping tablet replica
I20260808 16:52:29.061676 19994 raft_consensus.cc:2229] T a59d1aa03092471ab0ee5ed7598355d6 P 5f52c81c0e744fd195de3a1f27250198 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 16:52:29.061990 19321 tablet_service.cc:1514] Processing DeleteTablet for tablet a59d1aa03092471ab0ee5ed7598355d6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:52:29 UTC) from {username='oryx', principal='oryx/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:39848
I20260808 16:52:29.062075 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 17791
I20260808 16:52:29.070465 19994 raft_consensus.cc:2258] T a59d1aa03092471ab0ee5ed7598355d6 P 5f52c81c0e744fd195de3a1f27250198 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:52:29.070683 19996 tablet_replica.cc:330] T a59d1aa03092471ab0ee5ed7598355d6 P 005e71fe78a847c2b010b116c89a8609: stopping tablet replica
I20260808 16:52:29.070806 19996 raft_consensus.cc:2229] T a59d1aa03092471ab0ee5ed7598355d6 P 005e71fe78a847c2b010b116c89a8609 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 16:52:29.070941 19998 tablet_replica.cc:330] T a59d1aa03092471ab0ee5ed7598355d6 P 7b46262135f64e30b03b7b7cfa5e7044: stopping tablet replica
I20260808 16:52:29.071022 19998 raft_consensus.cc:2229] T a59d1aa03092471ab0ee5ed7598355d6 P 7b46262135f64e30b03b7b7cfa5e7044 [term 1 LEADER]: Raft consensus shutting down.
I20260808 16:52:29.071094 19998 raft_consensus.cc:2258] T a59d1aa03092471ab0ee5ed7598355d6 P 7b46262135f64e30b03b7b7cfa5e7044 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:52:29.071086 19994 ts_tablet_manager.cc:1905] T a59d1aa03092471ab0ee5ed7598355d6 P 5f52c81c0e744fd195de3a1f27250198: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:52:29.071213 19996 raft_consensus.cc:2258] T a59d1aa03092471ab0ee5ed7598355d6 P 005e71fe78a847c2b010b116c89a8609 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:52:29.072675 19994 ts_tablet_manager.cc:1918] T a59d1aa03092471ab0ee5ed7598355d6 P 5f52c81c0e744fd195de3a1f27250198: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 16:52:29.072741 19994 log.cc:1196] T a59d1aa03092471ab0ee5ed7598355d6 P 5f52c81c0e744fd195de3a1f27250198: Deleting WAL directory at /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/ts-2/wal/wals/a59d1aa03092471ab0ee5ed7598355d6
I20260808 16:52:29.072989 19994 ts_tablet_manager.cc:1939] T a59d1aa03092471ab0ee5ed7598355d6 P 5f52c81c0e744fd195de3a1f27250198: Deleting consensus metadata
I20260808 16:52:29.073752 19996 ts_tablet_manager.cc:1905] T a59d1aa03092471ab0ee5ed7598355d6 P 005e71fe78a847c2b010b116c89a8609: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:52:29.075243 19996 ts_tablet_manager.cc:1918] T a59d1aa03092471ab0ee5ed7598355d6 P 005e71fe78a847c2b010b116c89a8609: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 16:52:29.075320 19996 log.cc:1196] T a59d1aa03092471ab0ee5ed7598355d6 P 005e71fe78a847c2b010b116c89a8609: Deleting WAL directory at /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/ts-1/wal/wals/a59d1aa03092471ab0ee5ed7598355d6
I20260808 16:52:29.075615 19996 ts_tablet_manager.cc:1939] T a59d1aa03092471ab0ee5ed7598355d6 P 005e71fe78a847c2b010b116c89a8609: Deleting consensus metadata
W20260808 16:52:29.075969 19479 connection.cc:441] server connection from 127.0.0.1:45244 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45244 (request call id 1) could send its response
W20260808 16:52:29.076200 19700 connection.cc:535] server connection from 127.0.0.1:57440 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 16:52:29.083345 19998 ts_tablet_manager.cc:1905] T a59d1aa03092471ab0ee5ed7598355d6 P 7b46262135f64e30b03b7b7cfa5e7044: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:52:29.084240 18924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:29.077556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55326 (local address 127.8.24.61:45755)
0808 16:52:29.077597 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:29.077599 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:29.077713 (+ 114us) server_negotiation.cc:407] Connection header received
0808 16:52:29.077811 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:29.077812 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:29.077823 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:29.077853 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:29.078661 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:29.079401 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:29.080093 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:29.080292 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:29.080345 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:29.081491 (+ 1146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:29.081492 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:29.081493 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:29.081500 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:29.082935 (+ 1435us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:29.083307 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:29.083308 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:29.083309 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:29.083387 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:29.083679 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:29.083680 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:29.083680 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:29.083808 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:29.083878 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:29.084081 (+ 203us) server_negotiation.cc:299] Negotiation successful
0808 16:52:29.084156 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 16:52:29.084862 19998 ts_tablet_manager.cc:1918] T a59d1aa03092471ab0ee5ed7598355d6 P 7b46262135f64e30b03b7b7cfa5e7044: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 16:52:29.084918 19998 log.cc:1196] T a59d1aa03092471ab0ee5ed7598355d6 P 7b46262135f64e30b03b7b7cfa5e7044: Deleting WAL directory at /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/ts-0/wal/wals/a59d1aa03092471ab0ee5ed7598355d6
I20260808 16:52:29.085146 19998 ts_tablet_manager.cc:1939] T a59d1aa03092471ab0ee5ed7598355d6 P 7b46262135f64e30b03b7b7cfa5e7044: Deleting consensus metadata
W20260808 16:52:29.085471 19261 connection.cc:441] server connection from 127.0.0.1:39848 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:39848 (request call id 1) could send its response
I20260808 16:52:29.086074 19916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:29.077704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55822 (local address 127.8.24.60:39771)
0808 16:52:29.077743 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:29.077745 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:29.077830 (+ 85us) server_negotiation.cc:407] Connection header received
0808 16:52:29.077876 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:29.077877 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:29.077890 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:29.077945 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:29.079534 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:29.080300 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:29.081414 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:29.081735 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:29.081791 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:29.083095 (+ 1304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:29.083096 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:29.083096 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:29.083103 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:29.084910 (+ 1807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:29.085407 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:29.085409 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:29.085409 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:29.085472 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:29.085671 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:29.085673 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:29.085673 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:29.085777 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:29.085850 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:29.085961 (+ 111us) server_negotiation.cc:299] Negotiation successful
0808 16:52:29.086018 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 16:52:30.057897 20014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:30.057666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:46193 (local address 127.8.24.3:36379)
0808 16:52:30.057781 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:30.057858 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:46193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":60,"threads_started":1}
W20260808 16:52:30.058236 19893 heartbeater.cc:644] Failed to heartbeat to 127.8.24.62:46193 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.8.24.62:46193: connect: Connection refused (error 111)
I20260808 16:52:30.059710 20015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:30.059302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:46193 (local address 127.8.24.2:40275)
0808 16:52:30.059605 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:30.059668 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:46193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":49,"threads_started":1}
I20260808 16:52:30.059849 20016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:30.059531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:46193 (local address 127.8.24.1:49155)
0808 16:52:30.059702 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:30.059809 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:46193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":49,"threads_started":1}
W20260808 16:52:30.059873 19659 heartbeater.cc:644] Failed to heartbeat to 127.8.24.62:46193 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.8.24.62:46193: connect: Connection refused (error 111)
W20260808 16:52:30.060011 19440 heartbeater.cc:644] Failed to heartbeat to 127.8.24.62:46193 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.8.24.62:46193: connect: Connection refused (error 111)
I20260808 16:52:30.561404 20018 raft_consensus.cc:482] T 00000000000000000000000000000000 P ca479471edff409d957dde1bb92c4144 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 678b35993e9b49aca0f3ad2bba5edfbd)
I20260808 16:52:30.561491 20018 raft_consensus.cc:504] T 00000000000000000000000000000000 P ca479471edff409d957dde1bb92c4144 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "678b35993e9b49aca0f3ad2bba5edfbd" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 46193 } } peers { permanent_uuid: "ca479471edff409d957dde1bb92c4144" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 45755 } attrs { promote: false } } peers { permanent_uuid: "2a210ec3273f40d1b29270637e35af7b" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 39771 } attrs { promote: false } }
I20260808 16:52:30.561657 20018 leader_election.cc:288] T 00000000000000000000000000000000 P ca479471edff409d957dde1bb92c4144 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 678b35993e9b49aca0f3ad2bba5edfbd (127.8.24.62:46193), 2a210ec3273f40d1b29270637e35af7b (127.8.24.60:39771)
I20260808 16:52:30.561781 20019 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2a210ec3273f40d1b29270637e35af7b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 678b35993e9b49aca0f3ad2bba5edfbd)
I20260808 16:52:30.561915 20019 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2a210ec3273f40d1b29270637e35af7b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "678b35993e9b49aca0f3ad2bba5edfbd" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 46193 } } peers { permanent_uuid: "ca479471edff409d957dde1bb92c4144" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 45755 } attrs { promote: false } } peers { permanent_uuid: "2a210ec3273f40d1b29270637e35af7b" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 39771 } attrs { promote: false } }
I20260808 16:52:30.561959 20020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:30.561755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:46193 (local address 127.0.0.1:59492)
0808 16:52:30.561855 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:30.561912 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:46193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83,"thread_start_us":53,"threads_started":1}
W20260808 16:52:30.562085 18823 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.24.62:46193: connect: Connection refused (error 111)
I20260808 16:52:30.562196 20019 leader_election.cc:288] T 00000000000000000000000000000000 P 2a210ec3273f40d1b29270637e35af7b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 678b35993e9b49aca0f3ad2bba5edfbd (127.8.24.62:46193), ca479471edff409d957dde1bb92c4144 (127.8.24.61:45755)
I20260808 16:52:30.562481 20030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:30.562280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:46193 (local address 127.0.0.1:59496)
0808 16:52:30.562390 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:30.562444 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:46193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":44,"threads_started":1}
I20260808 16:52:30.562660 20020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:30.562576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:46193 (local address 127.0.0.1:59506)
0808 16:52:30.562606 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:30.562634 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:46193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
W20260808 16:52:30.562695 19126 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.24.62:46193: connect: Connection refused (error 111)
W20260808 16:52:30.562747 18823 leader_election.cc:334] T 00000000000000000000000000000000 P ca479471edff409d957dde1bb92c4144 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 678b35993e9b49aca0f3ad2bba5edfbd (127.8.24.62:46193): Network error: Client connection negotiation failed: client connection to 127.8.24.62:46193: connect: Connection refused (error 111)
I20260808 16:52:30.563079 20030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:30.562992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:46193 (local address 127.0.0.1:59510)
0808 16:52:30.563029 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:30.563055 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:46193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260808 16:52:30.563149 19126 leader_election.cc:334] T 00000000000000000000000000000000 P 2a210ec3273f40d1b29270637e35af7b [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 678b35993e9b49aca0f3ad2bba5edfbd (127.8.24.62:46193): Network error: Client connection negotiation failed: client connection to 127.8.24.62:46193: connect: Connection refused (error 111)
I20260808 16:52:30.565182 20021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:30.561905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.60:39771 (local address 127.0.0.1:55824)
0808 16:52:30.561999 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:30.562008 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:30.562285 (+ 277us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:30.562611 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:30.562612 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:30.562614 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:52:30.562675 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:30.562676 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:30.563545 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:30.563546 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:30.564380 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:30.564383 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:30.564951 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:30.564952 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:30.565020 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:30.565024 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 16:52:30.565069 (+ 45us) client_negotiation.cc:240] Negotiation successful
0808 16:52:30.565112 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":81,"thread_start_us":36,"threads_started":1}
I20260808 16:52:30.565196 20022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:30.562034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55824 (local address 127.8.24.60:39771)
0808 16:52:30.562451 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:30.562454 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:30.562462 (+ 8us) server_negotiation.cc:407] Connection header received
0808 16:52:30.562484 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:30.562484 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:30.562493 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:30.562581 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:52:30.562762 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:30.563464 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:30.564476 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:30.564880 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:30.564942 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:30.564954 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:30.565080 (+ 126us) server_negotiation.cc:299] Negotiation successful
0808 16:52:30.565124 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":55,"threads_started":1}
I20260808 16:52:30.565253 20029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:30.562281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.61:45755 (local address 127.0.0.1:55340)
0808 16:52:30.562395 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:30.562403 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:30.562452 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:30.562913 (+ 461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:30.562914 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:30.562915 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:52:30.562950 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:30.562952 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:30.563670 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:30.563671 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:30.564406 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:30.564409 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:30.564996 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:30.564997 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:30.565060 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:30.565063 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 16:52:30.565103 (+ 40us) client_negotiation.cc:240] Negotiation successful
0808 16:52:30.565148 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"mutex_wait_us":2,"thread_start_us":47,"threads_started":1}
I20260808 16:52:30.565258 20031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:30.562324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55340 (local address 127.8.24.61:45755)
0808 16:52:30.562550 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:30.562552 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:30.562558 (+ 6us) server_negotiation.cc:407] Connection header received
0808 16:52:30.562575 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:30.562576 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:30.562584 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:30.562902 (+ 318us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:52:30.563028 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:30.563587 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:30.564479 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:30.564884 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:30.564989 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:30.565005 (+ 16us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:30.565123 (+ 118us) server_negotiation.cc:299] Negotiation successful
0808 16:52:30.565165 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":97,"threads_started":1}
I20260808 16:52:30.565529 18860 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2a210ec3273f40d1b29270637e35af7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "ca479471edff409d957dde1bb92c4144" is_pre_election: true
I20260808 16:52:30.565557 19165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ca479471edff409d957dde1bb92c4144" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "2a210ec3273f40d1b29270637e35af7b" is_pre_election: true
I20260808 16:52:30.565625 18860 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ca479471edff409d957dde1bb92c4144 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a210ec3273f40d1b29270637e35af7b in term 1.
I20260808 16:52:30.565641 19165 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2a210ec3273f40d1b29270637e35af7b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca479471edff409d957dde1bb92c4144 in term 1.
I20260808 16:52:30.565763 19127 leader_election.cc:302] T 00000000000000000000000000000000 P 2a210ec3273f40d1b29270637e35af7b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2a210ec3273f40d1b29270637e35af7b, ca479471edff409d957dde1bb92c4144; no voters: 678b35993e9b49aca0f3ad2bba5edfbd
I20260808 16:52:30.565830 18823 leader_election.cc:302] T 00000000000000000000000000000000 P ca479471edff409d957dde1bb92c4144 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2a210ec3273f40d1b29270637e35af7b, ca479471edff409d957dde1bb92c4144; no voters: 678b35993e9b49aca0f3ad2bba5edfbd
I20260808 16:52:30.565869 20019 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2a210ec3273f40d1b29270637e35af7b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260808 16:52:30.565910 20019 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2a210ec3273f40d1b29270637e35af7b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 678b35993e9b49aca0f3ad2bba5edfbd)
I20260808 16:52:30.565907 20018 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ca479471edff409d957dde1bb92c4144 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260808 16:52:30.565937 20019 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2a210ec3273f40d1b29270637e35af7b [term 1 FOLLOWER]: Advancing to term 2
I20260808 16:52:30.565943 20018 raft_consensus.cc:482] T 00000000000000000000000000000000 P ca479471edff409d957dde1bb92c4144 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 678b35993e9b49aca0f3ad2bba5edfbd)
I20260808 16:52:30.565963 20018 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ca479471edff409d957dde1bb92c4144 [term 1 FOLLOWER]: Advancing to term 2
I20260808 16:52:30.566642 20019 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2a210ec3273f40d1b29270637e35af7b [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "678b35993e9b49aca0f3ad2bba5edfbd" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 46193 } } peers { permanent_uuid: "ca479471edff409d957dde1bb92c4144" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 45755 } attrs { promote: false } } peers { permanent_uuid: "2a210ec3273f40d1b29270637e35af7b" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 39771 } attrs { promote: false } }
I20260808 16:52:30.566670 20018 raft_consensus.cc:504] T 00000000000000000000000000000000 P ca479471edff409d957dde1bb92c4144 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "678b35993e9b49aca0f3ad2bba5edfbd" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 46193 } } peers { permanent_uuid: "ca479471edff409d957dde1bb92c4144" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 45755 } attrs { promote: false } } peers { permanent_uuid: "2a210ec3273f40d1b29270637e35af7b" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 39771 } attrs { promote: false } }
I20260808 16:52:30.566751 20019 leader_election.cc:288] T 00000000000000000000000000000000 P 2a210ec3273f40d1b29270637e35af7b [CANDIDATE]: Term 2 election: Requested vote from peers 678b35993e9b49aca0f3ad2bba5edfbd (127.8.24.62:46193), ca479471edff409d957dde1bb92c4144 (127.8.24.61:45755)
I20260808 16:52:30.566773 20018 leader_election.cc:288] T 00000000000000000000000000000000 P ca479471edff409d957dde1bb92c4144 [CANDIDATE]: Term 2 election: Requested vote from peers 678b35993e9b49aca0f3ad2bba5edfbd (127.8.24.62:46193), 2a210ec3273f40d1b29270637e35af7b (127.8.24.60:39771)
I20260808 16:52:30.566922 20029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:30.566835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:46193 (local address 127.0.0.1:59512)
0808 16:52:30.566870 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:30.566899 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:46193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260808 16:52:30.566959 18860 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2a210ec3273f40d1b29270637e35af7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "ca479471edff409d957dde1bb92c4144"
I20260808 16:52:30.567024 18860 raft_consensus.cc:2379] T 00000000000000000000000000000000 P ca479471edff409d957dde1bb92c4144 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 2a210ec3273f40d1b29270637e35af7b in current term 2: Already voted for candidate ca479471edff409d957dde1bb92c4144 in this term.
I20260808 16:52:30.567055 19165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ca479471edff409d957dde1bb92c4144" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "2a210ec3273f40d1b29270637e35af7b"
I20260808 16:52:30.567099 20021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:30.566928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:46193 (local address 127.0.0.1:59522)
0808 16:52:30.566998 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:30.567021 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:46193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260808 16:52:30.567117 19165 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 2a210ec3273f40d1b29270637e35af7b [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate ca479471edff409d957dde1bb92c4144 in current term 2: Already voted for candidate 2a210ec3273f40d1b29270637e35af7b in this term.
I20260808 16:52:30.567353 20029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:30.567263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:46193 (local address 127.0.0.1:59530)
0808 16:52:30.567310 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:30.567336 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:46193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260808 16:52:30.567436 20021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:30.567348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:46193 (local address 127.0.0.1:59534)
0808 16:52:30.567391 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:30.567417 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:46193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260808 16:52:30.567454 19126 leader_election.cc:334] T 00000000000000000000000000000000 P 2a210ec3273f40d1b29270637e35af7b [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 678b35993e9b49aca0f3ad2bba5edfbd (127.8.24.62:46193): Network error: Client connection negotiation failed: client connection to 127.8.24.62:46193: connect: Connection refused (error 111)
I20260808 16:52:30.567505 19126 leader_election.cc:302] T 00000000000000000000000000000000 P 2a210ec3273f40d1b29270637e35af7b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2a210ec3273f40d1b29270637e35af7b; no voters: 678b35993e9b49aca0f3ad2bba5edfbd, ca479471edff409d957dde1bb92c4144
W20260808 16:52:30.567514 18823 leader_election.cc:334] T 00000000000000000000000000000000 P ca479471edff409d957dde1bb92c4144 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 678b35993e9b49aca0f3ad2bba5edfbd (127.8.24.62:46193): Network error: Client connection negotiation failed: client connection to 127.8.24.62:46193: connect: Connection refused (error 111)
I20260808 16:52:30.567556 18823 leader_election.cc:302] T 00000000000000000000000000000000 P ca479471edff409d957dde1bb92c4144 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca479471edff409d957dde1bb92c4144; no voters: 2a210ec3273f40d1b29270637e35af7b, 678b35993e9b49aca0f3ad2bba5edfbd
I20260808 16:52:30.567618 20018 raft_consensus.cc:2726] T 00000000000000000000000000000000 P ca479471edff409d957dde1bb92c4144 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260808 16:52:30.567615 20019 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 2a210ec3273f40d1b29270637e35af7b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260808 16:52:31.058848 20037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:31.058626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:46193 (local address 127.8.24.3:36579)
0808 16:52:31.058769 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:31.058806 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:46193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":84,"threads_started":1}
I20260808 16:52:31.060309 20038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:31.060127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:46193 (local address 127.8.24.2:43337)
0808 16:52:31.060231 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:31.060267 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:46193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":49,"threads_started":1}
I20260808 16:52:31.060489 20039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:31.060268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:46193 (local address 127.8.24.1:41831)
0808 16:52:31.060415 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:31.060447 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:46193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":57,"threads_started":1}
I20260808 16:52:32.059427 20040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:32.059237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:46193 (local address 127.8.24.3:44147)
0808 16:52:32.059352 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:32.059388 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:46193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":61,"threads_started":1}
I20260808 16:52:32.060863 20041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:32.060657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:46193 (local address 127.8.24.2:56649)
0808 16:52:32.060782 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:32.060820 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:46193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":55,"threads_started":1}
I20260808 16:52:32.061048 20042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:32.060825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:46193 (local address 127.8.24.1:49039)
0808 16:52:32.060971 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:32.061006 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:46193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87,"thread_start_us":56,"threads_started":1}
I20260808 16:52:32.459851 20043 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2a210ec3273f40d1b29270637e35af7b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 16:52:32.459935 20043 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2a210ec3273f40d1b29270637e35af7b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "678b35993e9b49aca0f3ad2bba5edfbd" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 46193 } } peers { permanent_uuid: "ca479471edff409d957dde1bb92c4144" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 45755 } attrs { promote: false } } peers { permanent_uuid: "2a210ec3273f40d1b29270637e35af7b" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 39771 } attrs { promote: false } }
I20260808 16:52:32.460073 20043 leader_election.cc:288] T 00000000000000000000000000000000 P 2a210ec3273f40d1b29270637e35af7b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 678b35993e9b49aca0f3ad2bba5edfbd (127.8.24.62:46193), ca479471edff409d957dde1bb92c4144 (127.8.24.61:45755)
I20260808 16:52:32.460306 18860 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2a210ec3273f40d1b29270637e35af7b" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "ca479471edff409d957dde1bb92c4144" is_pre_election: true
I20260808 16:52:32.460394 18860 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ca479471edff409d957dde1bb92c4144 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a210ec3273f40d1b29270637e35af7b in term 2.
I20260808 16:52:32.460409 20044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:32.460190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:46193 (local address 127.0.0.1:59776)
0808 16:52:32.460327 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:32.460361 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:46193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":86,"threads_started":1}
I20260808 16:52:32.460580 19127 leader_election.cc:302] T 00000000000000000000000000000000 P 2a210ec3273f40d1b29270637e35af7b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a210ec3273f40d1b29270637e35af7b, ca479471edff409d957dde1bb92c4144; no voters:
I20260808 16:52:32.460688 20043 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2a210ec3273f40d1b29270637e35af7b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260808 16:52:32.460740 20043 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2a210ec3273f40d1b29270637e35af7b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 16:52:32.460786 20043 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2a210ec3273f40d1b29270637e35af7b [term 2 FOLLOWER]: Advancing to term 3
I20260808 16:52:32.460832 20044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:32.460731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:46193 (local address 127.0.0.1:59778)
0808 16:52:32.460781 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:32.460810 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:46193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260808 16:52:32.460927 19126 leader_election.cc:334] T 00000000000000000000000000000000 P 2a210ec3273f40d1b29270637e35af7b [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 678b35993e9b49aca0f3ad2bba5edfbd (127.8.24.62:46193): Network error: Client connection negotiation failed: client connection to 127.8.24.62:46193: connect: Connection refused (error 111)
I20260808 16:52:32.461572 20043 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2a210ec3273f40d1b29270637e35af7b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "678b35993e9b49aca0f3ad2bba5edfbd" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 46193 } } peers { permanent_uuid: "ca479471edff409d957dde1bb92c4144" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 45755 } attrs { promote: false } } peers { permanent_uuid: "2a210ec3273f40d1b29270637e35af7b" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 39771 } attrs { promote: false } }
I20260808 16:52:32.461690 20043 leader_election.cc:288] T 00000000000000000000000000000000 P 2a210ec3273f40d1b29270637e35af7b [CANDIDATE]: Term 3 election: Requested vote from peers 678b35993e9b49aca0f3ad2bba5edfbd (127.8.24.62:46193), ca479471edff409d957dde1bb92c4144 (127.8.24.61:45755)
I20260808 16:52:32.461869 20044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:32.461778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:46193 (local address 127.0.0.1:59792)
0808 16:52:32.461812 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:32.461844 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:46193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
I20260808 16:52:32.461859 18860 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2a210ec3273f40d1b29270637e35af7b" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "ca479471edff409d957dde1bb92c4144"
I20260808 16:52:32.461938 18860 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ca479471edff409d957dde1bb92c4144 [term 2 FOLLOWER]: Advancing to term 3
I20260808 16:52:32.462185 20044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:32.462090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:46193 (local address 127.0.0.1:59804)
0808 16:52:32.462133 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:32.462167 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:46193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260808 16:52:32.462256 19126 leader_election.cc:334] T 00000000000000000000000000000000 P 2a210ec3273f40d1b29270637e35af7b [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 678b35993e9b49aca0f3ad2bba5edfbd (127.8.24.62:46193): Network error: Client connection negotiation failed: client connection to 127.8.24.62:46193: connect: Connection refused (error 111)
I20260808 16:52:32.462666 18860 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ca479471edff409d957dde1bb92c4144 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a210ec3273f40d1b29270637e35af7b in term 3.
I20260808 16:52:32.462810 19127 leader_election.cc:302] T 00000000000000000000000000000000 P 2a210ec3273f40d1b29270637e35af7b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2a210ec3273f40d1b29270637e35af7b, ca479471edff409d957dde1bb92c4144; no voters: 678b35993e9b49aca0f3ad2bba5edfbd
I20260808 16:52:32.462880 20043 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2a210ec3273f40d1b29270637e35af7b [term 3 FOLLOWER]: Leader election won for term 3
I20260808 16:52:32.462990 20043 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2a210ec3273f40d1b29270637e35af7b [term 3 LEADER]: Becoming Leader. State: Replica: 2a210ec3273f40d1b29270637e35af7b, State: Running, Role: LEADER
I20260808 16:52:32.463068 20043 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2a210ec3273f40d1b29270637e35af7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "678b35993e9b49aca0f3ad2bba5edfbd" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 46193 } } peers { permanent_uuid: "ca479471edff409d957dde1bb92c4144" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 45755 } attrs { promote: false } } peers { permanent_uuid: "2a210ec3273f40d1b29270637e35af7b" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 39771 } attrs { promote: false } }
I20260808 16:52:32.463452 20046 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a210ec3273f40d1b29270637e35af7b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2a210ec3273f40d1b29270637e35af7b. Latest consensus state: current_term: 3 leader_uuid: "2a210ec3273f40d1b29270637e35af7b" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "678b35993e9b49aca0f3ad2bba5edfbd" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 46193 } } peers { permanent_uuid: "ca479471edff409d957dde1bb92c4144" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 45755 } attrs { promote: false } } peers { permanent_uuid: "2a210ec3273f40d1b29270637e35af7b" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 39771 } attrs { promote: false } } }
I20260808 16:52:32.463546 20046 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a210ec3273f40d1b29270637e35af7b [sys.catalog]: This master's current role is: LEADER
I20260808 16:52:32.463728 20048 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 16:52:32.464179 20048 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 16:52:32.464258 20048 catalog_manager.cc:1259] Loaded cluster ID: ddde3755643b4afbb76aa4f0f8896bc8
I20260808 16:52:32.464288 20048 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 16:52:32.464524 20048 catalog_manager.cc:1504] Loading token signing keys...
I20260808 16:52:32.464690 20048 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 2a210ec3273f40d1b29270637e35af7b: Loaded TSK: 0
I20260808 16:52:32.464841 20048 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 16:52:32.469861 19149 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55822:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 16:52:32.471412 19149 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 16:52:32.472252 20044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:32.472171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:46193 (local address 127.0.0.1:59824)
0808 16:52:32.472210 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:32.472233 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:46193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260808 16:52:32.472272 18860 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ca479471edff409d957dde1bb92c4144 [term 3 FOLLOWER]: Refusing update from remote peer 2a210ec3273f40d1b29270637e35af7b: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260808 16:52:32.472456 20046 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2a210ec3273f40d1b29270637e35af7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca479471edff409d957dde1bb92c4144" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 45755 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260808 16:52:32.472601 20044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:32.472508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:46193 (local address 127.0.0.1:59838)
0808 16:52:32.472554 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:32.472582 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:46193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260808 16:52:32.472676 19126 consensus_peers.cc:487] T 00000000000000000000000000000000 P 2a210ec3273f40d1b29270637e35af7b -> Peer 678b35993e9b49aca0f3ad2bba5edfbd (127.8.24.62:46193): Couldn't send request to peer 678b35993e9b49aca0f3ad2bba5edfbd. Status: Network error: Client connection negotiation failed: client connection to 127.8.24.62:46193: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260808 16:52:32.473009 20052 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca479471edff409d957dde1bb92c4144 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2a210ec3273f40d1b29270637e35af7b. Latest consensus state: current_term: 3 leader_uuid: "2a210ec3273f40d1b29270637e35af7b" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "678b35993e9b49aca0f3ad2bba5edfbd" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 46193 } } peers { permanent_uuid: "ca479471edff409d957dde1bb92c4144" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 45755 } attrs { promote: false } } peers { permanent_uuid: "2a210ec3273f40d1b29270637e35af7b" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 39771 } attrs { promote: false } } }
I20260808 16:52:32.473091 20052 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca479471edff409d957dde1bb92c4144 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 16:52:32.473507 20046 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a210ec3273f40d1b29270637e35af7b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "2a210ec3273f40d1b29270637e35af7b" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "678b35993e9b49aca0f3ad2bba5edfbd" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 46193 } } peers { permanent_uuid: "ca479471edff409d957dde1bb92c4144" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 45755 } attrs { promote: false } } peers { permanent_uuid: "2a210ec3273f40d1b29270637e35af7b" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 39771 } attrs { promote: false } } }
I20260808 16:52:32.473587 20046 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a210ec3273f40d1b29270637e35af7b [sys.catalog]: This master's current role is: LEADER
I20260808 16:52:32.473623 20053 mvcc.cc:205] Tried to move back new op lower bound from 7316307773324910592 to 7316307773289488384. Current Snapshot: MvccSnapshot[applied={T|T < 7316307773324910592}]
I20260808 16:52:32.473847 20052 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca479471edff409d957dde1bb92c4144 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "2a210ec3273f40d1b29270637e35af7b" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "678b35993e9b49aca0f3ad2bba5edfbd" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 46193 } } peers { permanent_uuid: "ca479471edff409d957dde1bb92c4144" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 45755 } attrs { promote: false } } peers { permanent_uuid: "2a210ec3273f40d1b29270637e35af7b" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 39771 } attrs { promote: false } } }
I20260808 16:52:32.473920 20052 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca479471edff409d957dde1bb92c4144 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 16:52:32.478457 20065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:32.475446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45248 (local address 127.8.24.2:38759)
0808 16:52:32.475709 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:32.475711 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:32.475719 (+ 8us) server_negotiation.cc:407] Connection header received
0808 16:52:32.475743 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:32.475744 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:32.475754 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:32.475820 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:52:32.475903 (+ 83us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:32.476602 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:32.477583 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:32.478119 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:32.478192 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:32.478204 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:32.478327 (+ 123us) server_negotiation.cc:299] Negotiation successful
0808 16:52:32.478388 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":53,"threads_started":1}
I20260808 16:52:32.478595 20044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:32.475111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.3:40919 (local address 127.0.0.1:57450)
0808 16:52:32.475302 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:32.475313 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:32.475370 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:32.475586 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:32.475588 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:32.475590 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:52:32.475636 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:32.475638 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:32.476563 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:32.476564 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:32.477693 (+ 1129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:32.477697 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:32.478406 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:32.478406 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:32.478457 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:32.478460 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 16:52:32.478505 (+ 45us) client_negotiation.cc:240] Negotiation successful
0808 16:52:32.478537 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"mutex_wait_us":64}
I20260808 16:52:32.478683 20061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:32.475196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57450 (local address 127.8.24.3:40919)
0808 16:52:32.475327 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:32.475329 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:32.475356 (+ 27us) server_negotiation.cc:407] Connection header received
0808 16:52:32.475423 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:32.475424 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:32.475434 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:32.475582 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:52:32.475720 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:32.476495 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:32.477768 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:32.478298 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:32.478410 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:32.478423 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:32.478524 (+ 101us) server_negotiation.cc:299] Negotiation successful
0808 16:52:32.478568 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":77,"threads_started":1}
I20260808 16:52:32.478771 20060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:32.475205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:38887 (local address 127.0.0.1:39878)
0808 16:52:32.475489 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:32.475496 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:32.475529 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:32.475788 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:32.475789 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:32.475790 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:52:32.475832 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:32.475834 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:32.476497 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:32.476498 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:32.477874 (+ 1376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:32.477876 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:32.478572 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:32.478572 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:32.478622 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:32.478626 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 16:52:32.478670 (+ 44us) client_negotiation.cc:240] Negotiation successful
0808 16:52:32.478712 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"mutex_wait_us":74,"thread_start_us":65,"threads_started":1}
I20260808 16:52:32.478953 20063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:32.475262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39878 (local address 127.8.24.1:38887)
0808 16:52:32.475642 (+ 380us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:32.475644 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:32.475652 (+ 8us) server_negotiation.cc:407] Connection header received
0808 16:52:32.475674 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:32.475675 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:32.475686 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:32.475742 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:52:32.475907 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:32.476420 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:32.477968 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:32.478479 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:32.478533 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:32.478545 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:32.478841 (+ 296us) server_negotiation.cc:299] Negotiation successful
0808 16:52:32.478890 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":42,"threads_started":1}
I20260808 16:52:32.479152 20058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:32.475177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.2:38759 (local address 127.0.0.1:45248)
0808 16:52:32.475360 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:32.475370 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:32.475420 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:32.475822 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:32.475822 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:32.475824 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:52:32.475852 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:32.475853 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:32.476672 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:32.476672 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:32.477517 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:32.477520 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:32.478198 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:32.478199 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:32.478258 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:32.478262 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 16:52:32.478315 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 16:52:32.479097 (+ 782us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"mutex_wait_us":4,"thread_start_us":93,"threads_started":1}
I20260808 16:52:32.479151 19321 tablet_service.cc:1467] Processing CreateTablet for tablet 4fa157e188e544cc983b73614340ad6c (DEFAULT_TABLE table=test-table [id=d2759668b28f4f6c86985c845a0f77d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:52:32.479190 19783 tablet_service.cc:1467] Processing CreateTablet for tablet 4fa157e188e544cc983b73614340ad6c (DEFAULT_TABLE table=test-table [id=d2759668b28f4f6c86985c845a0f77d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:52:32.479266 19321 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4fa157e188e544cc983b73614340ad6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:52:32.479308 19783 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4fa157e188e544cc983b73614340ad6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:52:32.480346 20066 tablet_bootstrap.cc:492] T 4fa157e188e544cc983b73614340ad6c P 5f52c81c0e744fd195de3a1f27250198: Bootstrap starting.
I20260808 16:52:32.480423 19545 tablet_service.cc:1467] Processing CreateTablet for tablet 4fa157e188e544cc983b73614340ad6c (DEFAULT_TABLE table=test-table [id=d2759668b28f4f6c86985c845a0f77d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:52:32.480541 19545 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4fa157e188e544cc983b73614340ad6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:52:32.480865 20066 tablet_bootstrap.cc:654] T 4fa157e188e544cc983b73614340ad6c P 5f52c81c0e744fd195de3a1f27250198: Neither blocks nor log segments found. Creating new log.
I20260808 16:52:32.481355 20066 tablet_bootstrap.cc:492] T 4fa157e188e544cc983b73614340ad6c P 5f52c81c0e744fd195de3a1f27250198: No bootstrap required, opened a new log
I20260808 16:52:32.481396 20066 ts_tablet_manager.cc:1397] T 4fa157e188e544cc983b73614340ad6c P 5f52c81c0e744fd195de3a1f27250198: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 16:52:32.481510 20066 raft_consensus.cc:348] T 4fa157e188e544cc983b73614340ad6c P 5f52c81c0e744fd195de3a1f27250198 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f52c81c0e744fd195de3a1f27250198" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40919 } } peers { permanent_uuid: "7b46262135f64e30b03b7b7cfa5e7044" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 38887 } } peers { permanent_uuid: "005e71fe78a847c2b010b116c89a8609" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 38759 } }
I20260808 16:52:32.481587 20071 tablet_bootstrap.cc:492] T 4fa157e188e544cc983b73614340ad6c P 7b46262135f64e30b03b7b7cfa5e7044: Bootstrap starting.
I20260808 16:52:32.481609 20066 raft_consensus.cc:374] T 4fa157e188e544cc983b73614340ad6c P 5f52c81c0e744fd195de3a1f27250198 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:52:32.481642 20066 raft_consensus.cc:729] T 4fa157e188e544cc983b73614340ad6c P 5f52c81c0e744fd195de3a1f27250198 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f52c81c0e744fd195de3a1f27250198, State: Initialized, Role: FOLLOWER
I20260808 16:52:32.481709 20075 tablet_bootstrap.cc:492] T 4fa157e188e544cc983b73614340ad6c P 005e71fe78a847c2b010b116c89a8609: Bootstrap starting.
I20260808 16:52:32.481726 20066 consensus_queue.cc:260] T 4fa157e188e544cc983b73614340ad6c P 5f52c81c0e744fd195de3a1f27250198 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f52c81c0e744fd195de3a1f27250198" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40919 } } peers { permanent_uuid: "7b46262135f64e30b03b7b7cfa5e7044" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 38887 } } peers { permanent_uuid: "005e71fe78a847c2b010b116c89a8609" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 38759 } }
I20260808 16:52:32.481918 20066 ts_tablet_manager.cc:1428] T 4fa157e188e544cc983b73614340ad6c P 5f52c81c0e744fd195de3a1f27250198: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 16:52:32.481997 20071 tablet_bootstrap.cc:654] T 4fa157e188e544cc983b73614340ad6c P 7b46262135f64e30b03b7b7cfa5e7044: Neither blocks nor log segments found. Creating new log.
I20260808 16:52:32.482102 20075 tablet_bootstrap.cc:654] T 4fa157e188e544cc983b73614340ad6c P 005e71fe78a847c2b010b116c89a8609: Neither blocks nor log segments found. Creating new log.
I20260808 16:52:32.482427 20040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:32.482203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:46193 (local address 127.8.24.3:52335)
0808 16:52:32.482372 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:32.482401 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:46193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153}
I20260808 16:52:32.482537 20071 tablet_bootstrap.cc:492] T 4fa157e188e544cc983b73614340ad6c P 7b46262135f64e30b03b7b7cfa5e7044: No bootstrap required, opened a new log
I20260808 16:52:32.482577 20071 ts_tablet_manager.cc:1397] T 4fa157e188e544cc983b73614340ad6c P 7b46262135f64e30b03b7b7cfa5e7044: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 16:52:32.482596 20075 tablet_bootstrap.cc:492] T 4fa157e188e544cc983b73614340ad6c P 005e71fe78a847c2b010b116c89a8609: No bootstrap required, opened a new log
I20260808 16:52:32.482630 20075 ts_tablet_manager.cc:1397] T 4fa157e188e544cc983b73614340ad6c P 005e71fe78a847c2b010b116c89a8609: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 16:52:32.482698 20071 raft_consensus.cc:348] T 4fa157e188e544cc983b73614340ad6c P 7b46262135f64e30b03b7b7cfa5e7044 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f52c81c0e744fd195de3a1f27250198" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40919 } } peers { permanent_uuid: "7b46262135f64e30b03b7b7cfa5e7044" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 38887 } } peers { permanent_uuid: "005e71fe78a847c2b010b116c89a8609" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 38759 } }
I20260808 16:52:32.482757 20071 raft_consensus.cc:374] T 4fa157e188e544cc983b73614340ad6c P 7b46262135f64e30b03b7b7cfa5e7044 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:52:32.482726 20075 raft_consensus.cc:348] T 4fa157e188e544cc983b73614340ad6c P 005e71fe78a847c2b010b116c89a8609 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f52c81c0e744fd195de3a1f27250198" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40919 } } peers { permanent_uuid: "7b46262135f64e30b03b7b7cfa5e7044" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 38887 } } peers { permanent_uuid: "005e71fe78a847c2b010b116c89a8609" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 38759 } }
I20260808 16:52:32.482775 20071 raft_consensus.cc:729] T 4fa157e188e544cc983b73614340ad6c P 7b46262135f64e30b03b7b7cfa5e7044 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b46262135f64e30b03b7b7cfa5e7044, State: Initialized, Role: FOLLOWER
I20260808 16:52:32.482779 20075 raft_consensus.cc:374] T 4fa157e188e544cc983b73614340ad6c P 005e71fe78a847c2b010b116c89a8609 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:52:32.482797 20075 raft_consensus.cc:729] T 4fa157e188e544cc983b73614340ad6c P 005e71fe78a847c2b010b116c89a8609 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 005e71fe78a847c2b010b116c89a8609, State: Initialized, Role: FOLLOWER
I20260808 16:52:32.482832 20075 consensus_queue.cc:260] T 4fa157e188e544cc983b73614340ad6c P 005e71fe78a847c2b010b116c89a8609 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f52c81c0e744fd195de3a1f27250198" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40919 } } peers { permanent_uuid: "7b46262135f64e30b03b7b7cfa5e7044" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 38887 } } peers { permanent_uuid: "005e71fe78a847c2b010b116c89a8609" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 38759 } }
I20260808 16:52:32.482834 20071 consensus_queue.cc:260] T 4fa157e188e544cc983b73614340ad6c P 7b46262135f64e30b03b7b7cfa5e7044 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f52c81c0e744fd195de3a1f27250198" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40919 } } peers { permanent_uuid: "7b46262135f64e30b03b7b7cfa5e7044" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 38887 } } peers { permanent_uuid: "005e71fe78a847c2b010b116c89a8609" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 38759 } }
I20260808 16:52:32.482996 20071 ts_tablet_manager.cc:1428] T 4fa157e188e544cc983b73614340ad6c P 7b46262135f64e30b03b7b7cfa5e7044: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 16:52:32.483008 20075 ts_tablet_manager.cc:1428] T 4fa157e188e544cc983b73614340ad6c P 005e71fe78a847c2b010b116c89a8609: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:32.483255 20041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:32.483126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:46193 (local address 127.8.24.2:36515)
0808 16:52:32.483204 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:32.483229 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:46193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260808 16:52:32.483397 20042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:32.483307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:46193 (local address 127.8.24.1:50859)
0808 16:52:32.483342 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:32.483371 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:46193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260808 16:52:32.483615 20041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:32.483541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:46193 (local address 127.8.24.2:50069)
0808 16:52:32.483573 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:32.483593 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:46193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260808 16:52:32.483693 19658 heartbeater.cc:503] Master 127.8.24.60:39771 was elected leader, sending a full tablet report...
I20260808 16:52:32.483953 19891 heartbeater.cc:503] Master 127.8.24.60:39771 was elected leader, sending a full tablet report...
I20260808 16:52:32.484150 20040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:32.484073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:46193 (local address 127.8.24.3:53099)
0808 16:52:32.484101 (+ 28us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:32.484125 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:46193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
I20260808 16:52:32.484436 19439 heartbeater.cc:503] Master 127.8.24.60:39771 was elected leader, sending a full tablet report...
I20260808 16:52:32.484635 20042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:32.484544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:46193 (local address 127.8.24.1:49997)
0808 16:52:32.484575 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:32.484601 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:46193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
I20260808 16:52:32.787715 20083 raft_consensus.cc:482] T 4fa157e188e544cc983b73614340ad6c P 7b46262135f64e30b03b7b7cfa5e7044 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 16:52:32.787818 20083 raft_consensus.cc:504] T 4fa157e188e544cc983b73614340ad6c P 7b46262135f64e30b03b7b7cfa5e7044 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f52c81c0e744fd195de3a1f27250198" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40919 } } peers { permanent_uuid: "7b46262135f64e30b03b7b7cfa5e7044" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 38887 } } peers { permanent_uuid: "005e71fe78a847c2b010b116c89a8609" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 38759 } }
I20260808 16:52:32.787990 20083 leader_election.cc:288] T 4fa157e188e544cc983b73614340ad6c P 7b46262135f64e30b03b7b7cfa5e7044 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5f52c81c0e744fd195de3a1f27250198 (127.8.24.3:40919), 005e71fe78a847c2b010b116c89a8609 (127.8.24.2:38759)
I20260808 16:52:32.788241 19818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fa157e188e544cc983b73614340ad6c" candidate_uuid: "7b46262135f64e30b03b7b7cfa5e7044" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f52c81c0e744fd195de3a1f27250198" is_pre_election: true
I20260808 16:52:32.788235 19573 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fa157e188e544cc983b73614340ad6c" candidate_uuid: "7b46262135f64e30b03b7b7cfa5e7044" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "005e71fe78a847c2b010b116c89a8609" is_pre_election: true
I20260808 16:52:32.788348 19818 raft_consensus.cc:2454] T 4fa157e188e544cc983b73614340ad6c P 5f52c81c0e744fd195de3a1f27250198 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7b46262135f64e30b03b7b7cfa5e7044 in term 0.
I20260808 16:52:32.788349 19573 raft_consensus.cc:2454] T 4fa157e188e544cc983b73614340ad6c P 005e71fe78a847c2b010b116c89a8609 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7b46262135f64e30b03b7b7cfa5e7044 in term 0.
I20260808 16:52:32.788511 19263 leader_election.cc:302] T 4fa157e188e544cc983b73614340ad6c P 7b46262135f64e30b03b7b7cfa5e7044 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 005e71fe78a847c2b010b116c89a8609, 7b46262135f64e30b03b7b7cfa5e7044; no voters:
I20260808 16:52:32.788645 20083 raft_consensus.cc:2781] T 4fa157e188e544cc983b73614340ad6c P 7b46262135f64e30b03b7b7cfa5e7044 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 16:52:32.788697 20083 raft_consensus.cc:482] T 4fa157e188e544cc983b73614340ad6c P 7b46262135f64e30b03b7b7cfa5e7044 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 16:52:32.788712 20083 raft_consensus.cc:3037] T 4fa157e188e544cc983b73614340ad6c P 7b46262135f64e30b03b7b7cfa5e7044 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:52:32.789417 20083 raft_consensus.cc:504] T 4fa157e188e544cc983b73614340ad6c P 7b46262135f64e30b03b7b7cfa5e7044 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f52c81c0e744fd195de3a1f27250198" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40919 } } peers { permanent_uuid: "7b46262135f64e30b03b7b7cfa5e7044" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 38887 } } peers { permanent_uuid: "005e71fe78a847c2b010b116c89a8609" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 38759 } }
I20260808 16:52:32.789536 20083 leader_election.cc:288] T 4fa157e188e544cc983b73614340ad6c P 7b46262135f64e30b03b7b7cfa5e7044 [CANDIDATE]: Term 1 election: Requested vote from peers 5f52c81c0e744fd195de3a1f27250198 (127.8.24.3:40919), 005e71fe78a847c2b010b116c89a8609 (127.8.24.2:38759)
I20260808 16:52:32.789660 19818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fa157e188e544cc983b73614340ad6c" candidate_uuid: "7b46262135f64e30b03b7b7cfa5e7044" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f52c81c0e744fd195de3a1f27250198"
I20260808 16:52:32.789728 19818 raft_consensus.cc:3037] T 4fa157e188e544cc983b73614340ad6c P 5f52c81c0e744fd195de3a1f27250198 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:52:32.789680 19573 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fa157e188e544cc983b73614340ad6c" candidate_uuid: "7b46262135f64e30b03b7b7cfa5e7044" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "005e71fe78a847c2b010b116c89a8609"
I20260808 16:52:32.789764 19573 raft_consensus.cc:3037] T 4fa157e188e544cc983b73614340ad6c P 005e71fe78a847c2b010b116c89a8609 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:52:32.790290 19818 raft_consensus.cc:2454] T 4fa157e188e544cc983b73614340ad6c P 5f52c81c0e744fd195de3a1f27250198 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7b46262135f64e30b03b7b7cfa5e7044 in term 1.
I20260808 16:52:32.790426 19573 raft_consensus.cc:2454] T 4fa157e188e544cc983b73614340ad6c P 005e71fe78a847c2b010b116c89a8609 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7b46262135f64e30b03b7b7cfa5e7044 in term 1.
I20260808 16:52:32.790486 19259 leader_election.cc:302] T 4fa157e188e544cc983b73614340ad6c P 7b46262135f64e30b03b7b7cfa5e7044 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5f52c81c0e744fd195de3a1f27250198, 7b46262135f64e30b03b7b7cfa5e7044; no voters:
I20260808 16:52:32.790568 20083 raft_consensus.cc:2781] T 4fa157e188e544cc983b73614340ad6c P 7b46262135f64e30b03b7b7cfa5e7044 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:52:32.790716 20083 raft_consensus.cc:686] T 4fa157e188e544cc983b73614340ad6c P 7b46262135f64e30b03b7b7cfa5e7044 [term 1 LEADER]: Becoming Leader. State: Replica: 7b46262135f64e30b03b7b7cfa5e7044, State: Running, Role: LEADER
I20260808 16:52:32.790768 20083 consensus_queue.cc:237] T 4fa157e188e544cc983b73614340ad6c P 7b46262135f64e30b03b7b7cfa5e7044 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f52c81c0e744fd195de3a1f27250198" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40919 } } peers { permanent_uuid: "7b46262135f64e30b03b7b7cfa5e7044" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 38887 } } peers { permanent_uuid: "005e71fe78a847c2b010b116c89a8609" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 38759 } }
I20260808 16:52:32.791055 20042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:32.790941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:46193 (local address 127.8.24.1:41943)
0808 16:52:32.790985 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:32.791023 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:46193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260808 16:52:32.791311 19150 catalog_manager.cc:5495] T 4fa157e188e544cc983b73614340ad6c P 7b46262135f64e30b03b7b7cfa5e7044 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7b46262135f64e30b03b7b7cfa5e7044 (127.8.24.1). New cstate: current_term: 1 leader_uuid: "7b46262135f64e30b03b7b7cfa5e7044" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f52c81c0e744fd195de3a1f27250198" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7b46262135f64e30b03b7b7cfa5e7044" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 38887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "005e71fe78a847c2b010b116c89a8609" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 38759 } health_report { overall_health: UNKNOWN } } }
I20260808 16:52:32.828383 20063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:32.826494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39884 (local address 127.8.24.1:38887)
0808 16:52:32.826541 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:32.826543 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:32.826552 (+ 9us) server_negotiation.cc:407] Connection header received
0808 16:52:32.826608 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:32.826609 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:32.826619 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:32.826662 (+ 43us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 16:52:32.826761 (+ 99us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:32.827211 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:32.827899 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:32.828081 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:32.828129 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:32.828211 (+ 82us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 16:52:32.828234 (+ 23us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 16:52:32.828273 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:32.828311 (+ 38us) server_negotiation.cc:299] Negotiation successful
0808 16:52:32.828328 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 16:52:32.829157 19573 raft_consensus.cc:1261] T 4fa157e188e544cc983b73614340ad6c P 005e71fe78a847c2b010b116c89a8609 [term 1 FOLLOWER]: Refusing update from remote peer 7b46262135f64e30b03b7b7cfa5e7044: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:52:32.829170 19818 raft_consensus.cc:1261] T 4fa157e188e544cc983b73614340ad6c P 5f52c81c0e744fd195de3a1f27250198 [term 1 FOLLOWER]: Refusing update from remote peer 7b46262135f64e30b03b7b7cfa5e7044: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:52:32.829380 20084 consensus_queue.cc:1035] T 4fa157e188e544cc983b73614340ad6c P 7b46262135f64e30b03b7b7cfa5e7044 [LEADER]: Connected to new peer: Peer: permanent_uuid: "005e71fe78a847c2b010b116c89a8609" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 38759 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:52:32.829491 20083 consensus_queue.cc:1035] T 4fa157e188e544cc983b73614340ad6c P 7b46262135f64e30b03b7b7cfa5e7044 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f52c81c0e744fd195de3a1f27250198" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:52:32.830071 20041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:32.829957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:46193 (local address 127.8.24.2:33961)
0808 16:52:32.830014 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:32.830042 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:46193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260808 16:52:32.830790 20040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:32.830687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:46193 (local address 127.8.24.3:57325)
0808 16:52:32.830724 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:32.830761 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:46193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260808 16:52:32.831420 20042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:32.831293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:46193 (local address 127.8.24.1:57687)
0808 16:52:32.831366 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:32.831392 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:46193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260808 16:52:32.832073 20041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:32.831943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:46193 (local address 127.8.24.2:60505)
0808 16:52:32.831997 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:32.832045 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:46193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260808 16:52:32.832410 20040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:32.832314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:46193 (local address 127.8.24.3:53903)
0808 16:52:32.832350 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:32.832380 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:46193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
I20260808 16:52:32.833125 19150 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55822:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 16:52:32.833217 19150 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55822:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 16:52:32.834359 19150 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2a210ec3273f40d1b29270637e35af7b: Sending DeleteTablet for 3 replicas of tablet 4fa157e188e544cc983b73614340ad6c
I20260808 16:52:32.834619 19545 tablet_service.cc:1514] Processing DeleteTablet for tablet 4fa157e188e544cc983b73614340ad6c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:52:32 UTC) from {username='oryx', principal='oryx/127.8.24.60@KRBTEST.COM'} at 127.0.0.1:45248
I20260808 16:52:32.834645 19783 tablet_service.cc:1514] Processing DeleteTablet for tablet 4fa157e188e544cc983b73614340ad6c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:52:32 UTC) from {username='oryx', principal='oryx/127.8.24.60@KRBTEST.COM'} at 127.0.0.1:57450
I20260808 16:52:32.834803 19321 tablet_service.cc:1514] Processing DeleteTablet for tablet 4fa157e188e544cc983b73614340ad6c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:52:32 UTC) from {username='oryx', principal='oryx/127.8.24.60@KRBTEST.COM'} at 127.0.0.1:39878
I20260808 16:52:32.834828 20108 tablet_replica.cc:330] T 4fa157e188e544cc983b73614340ad6c P 005e71fe78a847c2b010b116c89a8609: stopping tablet replica
I20260808 16:52:32.834828 20106 tablet_replica.cc:330] T 4fa157e188e544cc983b73614340ad6c P 5f52c81c0e744fd195de3a1f27250198: stopping tablet replica
I20260808 16:52:32.834880 20106 raft_consensus.cc:2229] T 4fa157e188e544cc983b73614340ad6c P 5f52c81c0e744fd195de3a1f27250198 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 16:52:32.834880 20108 raft_consensus.cc:2229] T 4fa157e188e544cc983b73614340ad6c P 005e71fe78a847c2b010b116c89a8609 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 16:52:32.834916 20106 raft_consensus.cc:2258] T 4fa157e188e544cc983b73614340ad6c P 5f52c81c0e744fd195de3a1f27250198 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:52:32.834916 20108 raft_consensus.cc:2258] T 4fa157e188e544cc983b73614340ad6c P 005e71fe78a847c2b010b116c89a8609 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:52:32.834969 20109 tablet_replica.cc:330] T 4fa157e188e544cc983b73614340ad6c P 7b46262135f64e30b03b7b7cfa5e7044: stopping tablet replica
I20260808 16:52:32.835017 20109 raft_consensus.cc:2229] T 4fa157e188e544cc983b73614340ad6c P 7b46262135f64e30b03b7b7cfa5e7044 [term 1 LEADER]: Raft consensus shutting down.
I20260808 16:52:32.835067 20109 raft_consensus.cc:2258] T 4fa157e188e544cc983b73614340ad6c P 7b46262135f64e30b03b7b7cfa5e7044 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:52:32.835289 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 19232
I20260808 16:52:32.835342 20109 ts_tablet_manager.cc:1905] T 4fa157e188e544cc983b73614340ad6c P 7b46262135f64e30b03b7b7cfa5e7044: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:52:32.835378 20106 ts_tablet_manager.cc:1905] T 4fa157e188e544cc983b73614340ad6c P 5f52c81c0e744fd195de3a1f27250198: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:52:32.836817 20106 ts_tablet_manager.cc:1918] T 4fa157e188e544cc983b73614340ad6c P 5f52c81c0e744fd195de3a1f27250198: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 16:52:32.836853 20106 log.cc:1196] T 4fa157e188e544cc983b73614340ad6c P 5f52c81c0e744fd195de3a1f27250198: Deleting WAL directory at /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/ts-2/wal/wals/4fa157e188e544cc983b73614340ad6c
I20260808 16:52:32.836992 20106 ts_tablet_manager.cc:1939] T 4fa157e188e544cc983b73614340ad6c P 5f52c81c0e744fd195de3a1f27250198: Deleting consensus metadata
I20260808 16:52:32.837276 19124 catalog_manager.cc:4841] TS 5f52c81c0e744fd195de3a1f27250198 (127.8.24.3:40919): tablet 4fa157e188e544cc983b73614340ad6c (table test-table [id=d2759668b28f4f6c86985c845a0f77d0]) successfully deleted
I20260808 16:52:32.842775 20108 ts_tablet_manager.cc:1905] T 4fa157e188e544cc983b73614340ad6c P 005e71fe78a847c2b010b116c89a8609: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:52:32.843896 20108 ts_tablet_manager.cc:1918] T 4fa157e188e544cc983b73614340ad6c P 005e71fe78a847c2b010b116c89a8609: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 16:52:32.843946 20108 log.cc:1196] T 4fa157e188e544cc983b73614340ad6c P 005e71fe78a847c2b010b116c89a8609: Deleting WAL directory at /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786207919384338-8288-0/minicluster-data/ts-1/wal/wals/4fa157e188e544cc983b73614340ad6c
I20260808 16:52:32.844115 20108 ts_tablet_manager.cc:1939] T 4fa157e188e544cc983b73614340ad6c P 005e71fe78a847c2b010b116c89a8609: Deleting consensus metadata
I20260808 16:52:32.844399 19128 catalog_manager.cc:4841] TS 005e71fe78a847c2b010b116c89a8609 (127.8.24.2:38759): tablet 4fa157e188e544cc983b73614340ad6c (table test-table [id=d2759668b28f4f6c86985c845a0f77d0]) successfully deleted
I20260808 16:52:32.847632 20058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:32.847538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:38887 (local address 127.0.0.1:39890)
0808 16:52:32.847584 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:32.847606 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.1:38887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260808 16:52:32.847667 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 19454
W20260808 16:52:32.847788 19124 catalog_manager.cc:4584] TS 7b46262135f64e30b03b7b7cfa5e7044 (127.8.24.1:38887): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4fa157e188e544cc983b73614340ad6c: Network error: Client connection negotiation failed: client connection to 127.8.24.1:38887: connect: Connection refused (error 111)
I20260808 16:52:32.856190 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 19666
I20260808 16:52:32.872164 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 18627
I20260808 16:52:32.884263 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 18928
2026-08-08T16:52:32Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (7842 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 16:52:32 dist-test-slave-6zbq krb5kdc[20112](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 16:52:32 dist-test-slave-6zbq krb5kdc[20112](info): set up 2 sockets
Aug 08 16:52:32 dist-test-slave-6zbq krb5kdc[20112](info): commencing operation
krb5kdc: starting...
W20260808 16:52:34.012821 8288 mini_kdc.cc:121] Time spent starting KDC: real 1.105s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/krb5kdc/test-user.keytab.
Aug 08 16:52:34 dist-test-slave-6zbq krb5kdc[20112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207954, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T16:52:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T16:52:34Z Disabled control of system clock
WARNING: no policy specified for oryx/127.8.24.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.24.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.24.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/krb5kdc/oryx_127.8.24.62.keytab.
Entry for principal oryx/127.8.24.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/krb5kdc/oryx_127.8.24.62.keytab.
I20260808 16:52:34.118687 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:39641
--webserver_interface=127.8.24.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/krb5kdc/oryx_127.8.24.62.keytab
--principal=oryx/127.8.24.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:38953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:39641
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:52:34.196069 20135 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:52:34.196245 20135 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:52:34.196265 20135 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:52:34.197371 20135 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 16:52:34.197409 20135 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 16:52:34.197422 20135 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:52:34.197432 20135 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:52:34.197456 20135 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 16:52:34.197476 20135 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 16:52:34.198959 20135 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:38953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:39641
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/krb5kdc/oryx_127.8.24.62.keytab
--principal=oryx/127.8.24.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:39641
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.24.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20135
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:52:34.199170 20135 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:52:34.199379 20135 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:52:34.204284 20141 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:34.204283 20146 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:34.204448 20135 server_base.cc:1034] running on GCE node
W20260808 16:52:34.204473 20142 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:34.204813 20135 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:52:34.205081 20135 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:52:34.206218 20135 hybrid_clock.cc:648] HybridClock initialized: now 1786207954206204 us; error 29 us; skew 500 ppm
Aug 08 16:52:34 dist-test-slave-6zbq krb5kdc[20112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207954, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.24.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:52:34.208539 20135 init.cc:377] Logged in from keytab as oryx/127.8.24.62@KRBTEST.COM (short username oryx)
I20260808 16:52:34.208968 20135 webserver.cc:460] Webserver started at http://127.8.24.62:38391/ using document root <none> and password file <none>
I20260808 16:52:34.209163 20135 fs_manager.cc:359] Metadata directory not provided
I20260808 16:52:34.209209 20135 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:52:34.209337 20135 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:52:34.210389 20135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/minicluster-data/master-0/data/instance:
uuid: "e63efa3dec1242f980a4d3123bd3f739"
format_stamp: "Formatted at 2026-08-08 16:52:34 on dist-test-slave-6zbq"
server_key: "95337098f9aa209630463e4320c32f92"
server_key_iv: "dffc3fd2217dfbe6101ef56b021b734f"
server_key_version: "encryptionkey@0"
I20260808 16:52:34.210729 20135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/minicluster-data/master-0/wal/instance:
uuid: "e63efa3dec1242f980a4d3123bd3f739"
format_stamp: "Formatted at 2026-08-08 16:52:34 on dist-test-slave-6zbq"
server_key: "95337098f9aa209630463e4320c32f92"
server_key_iv: "dffc3fd2217dfbe6101ef56b021b734f"
server_key_version: "encryptionkey@0"
I20260808 16:52:34.211941 20135 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 16:52:34.212603 20158 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:34.212751 20135 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:34.212834 20135 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/minicluster-data/master-0/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/minicluster-data/master-0/wal
uuid: "e63efa3dec1242f980a4d3123bd3f739"
format_stamp: "Formatted at 2026-08-08 16:52:34 on dist-test-slave-6zbq"
server_key: "95337098f9aa209630463e4320c32f92"
server_key_iv: "dffc3fd2217dfbe6101ef56b021b734f"
server_key_version: "encryptionkey@0"
I20260808 16:52:34.212901 20135 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:52:34.241772 20135 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:52:34.243597 20135 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:52:34.243731 20135 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:52:34.248027 20135 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.62:39641
I20260808 16:52:34.248080 20256 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.62:39641 every 8 connection(s)
I20260808 16:52:34.248399 20135 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
I20260808 16:52:34.248997 20258 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:52:34.251225 20258 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e63efa3dec1242f980a4d3123bd3f739: Bootstrap starting.
I20260808 16:52:34.251791 20258 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e63efa3dec1242f980a4d3123bd3f739: Neither blocks nor log segments found. Creating new log.
I20260808 16:52:34.252053 20258 log.cc:824] T 00000000000000000000000000000000 P e63efa3dec1242f980a4d3123bd3f739: Log is configured to *not* fsync() on all Append() calls
I20260808 16:52:34.252691 20258 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e63efa3dec1242f980a4d3123bd3f739: No bootstrap required, opened a new log
I20260808 16:52:34.252856 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 20135
I20260808 16:52:34.252943 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/minicluster-data/master-0/wal/instance
I20260808 16:52:34.253142 8288 external_mini_cluster.cc:1428] Setting key bf195ab2d3800abc1a6c14690ae905b8
I20260808 16:52:34.255071 20258 raft_consensus.cc:348] T 00000000000000000000000000000000 P e63efa3dec1242f980a4d3123bd3f739 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e63efa3dec1242f980a4d3123bd3f739" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 39641 } }
I20260808 16:52:34.255224 20258 raft_consensus.cc:374] T 00000000000000000000000000000000 P e63efa3dec1242f980a4d3123bd3f739 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:52:34.255261 20258 raft_consensus.cc:729] T 00000000000000000000000000000000 P e63efa3dec1242f980a4d3123bd3f739 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e63efa3dec1242f980a4d3123bd3f739, State: Initialized, Role: FOLLOWER
I20260808 16:52:34.255389 20258 consensus_queue.cc:260] T 00000000000000000000000000000000 P e63efa3dec1242f980a4d3123bd3f739 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e63efa3dec1242f980a4d3123bd3f739" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 39641 } }
I20260808 16:52:34.255465 20258 raft_consensus.cc:388] T 00000000000000000000000000000000 P e63efa3dec1242f980a4d3123bd3f739 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 16:52:34.255502 20258 raft_consensus.cc:482] T 00000000000000000000000000000000 P e63efa3dec1242f980a4d3123bd3f739 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 16:52:34.255550 20258 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e63efa3dec1242f980a4d3123bd3f739 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:52:34.256161 20258 raft_consensus.cc:504] T 00000000000000000000000000000000 P e63efa3dec1242f980a4d3123bd3f739 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e63efa3dec1242f980a4d3123bd3f739" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 39641 } }
I20260808 16:52:34.256259 20258 leader_election.cc:302] T 00000000000000000000000000000000 P e63efa3dec1242f980a4d3123bd3f739 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e63efa3dec1242f980a4d3123bd3f739; no voters:
I20260808 16:52:34.256400 20258 leader_election.cc:288] T 00000000000000000000000000000000 P e63efa3dec1242f980a4d3123bd3f739 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 16:52:34.256513 20265 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e63efa3dec1242f980a4d3123bd3f739 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:52:34.256613 20265 raft_consensus.cc:686] T 00000000000000000000000000000000 P e63efa3dec1242f980a4d3123bd3f739 [term 1 LEADER]: Becoming Leader. State: Replica: e63efa3dec1242f980a4d3123bd3f739, State: Running, Role: LEADER
I20260808 16:52:34.256760 20258 sys_catalog.cc:564] T 00000000000000000000000000000000 P e63efa3dec1242f980a4d3123bd3f739 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 16:52:34.256770 20265 consensus_queue.cc:237] T 00000000000000000000000000000000 P e63efa3dec1242f980a4d3123bd3f739 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e63efa3dec1242f980a4d3123bd3f739" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 39641 } }
I20260808 16:52:34.257227 20266 sys_catalog.cc:455] T 00000000000000000000000000000000 P e63efa3dec1242f980a4d3123bd3f739 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e63efa3dec1242f980a4d3123bd3f739. Latest consensus state: current_term: 1 leader_uuid: "e63efa3dec1242f980a4d3123bd3f739" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e63efa3dec1242f980a4d3123bd3f739" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 39641 } } }
I20260808 16:52:34.257241 20267 sys_catalog.cc:455] T 00000000000000000000000000000000 P e63efa3dec1242f980a4d3123bd3f739 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e63efa3dec1242f980a4d3123bd3f739" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e63efa3dec1242f980a4d3123bd3f739" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 39641 } } }
I20260808 16:52:34.257416 20266 sys_catalog.cc:458] T 00000000000000000000000000000000 P e63efa3dec1242f980a4d3123bd3f739 [sys.catalog]: This master's current role is: LEADER
I20260808 16:52:34.257427 20267 sys_catalog.cc:458] T 00000000000000000000000000000000 P e63efa3dec1242f980a4d3123bd3f739 [sys.catalog]: This master's current role is: LEADER
W20260808 16:52:34.258673 20289 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e63efa3dec1242f980a4d3123bd3f739: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 16:52:34.258770 20289 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 16:52:34.258988 20278 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 16:52:34.259166 20278 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 08 16:52:34 dist-test-slave-6zbq krb5kdc[20112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207954, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM
I20260808 16:52:34.260942 20278 catalog_manager.cc:1347] Generated new cluster ID: 2513524a754b4e969cc3ee77a06cf56a
I20260808 16:52:34.261009 20278 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 16:52:34.265558 20264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:34.253614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55388 (local address 127.8.24.62:39641)
0808 16:52:34.253769 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:34.253778 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:34.253815 (+ 37us) server_negotiation.cc:407] Connection header received
0808 16:52:34.255043 (+ 1228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:34.255050 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:34.255095 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:34.255204 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:34.256004 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:34.257195 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:34.258233 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:34.258737 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:34.258833 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:34.261340 (+ 2507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:34.261344 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:34.261344 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:34.261356 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:34.263998 (+ 2642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:34.264358 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:34.264362 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:34.264362 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:34.264491 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:34.264667 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:34.264668 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:34.264669 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:34.264980 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:34.265138 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:34.265218 (+ 80us) server_negotiation.cc:299] Negotiation successful
0808 16:52:34.265360 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":67,"threads_started":1}
I20260808 16:52:34.269042 20278 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 16:52:34.269781 20278 catalog_manager.cc:1504] Loading token signing keys...
I20260808 16:52:34.279062 20278 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e63efa3dec1242f980a4d3123bd3f739: Generated new TSK 0
I20260808 16:52:34.279224 20278 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.8.24.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.24.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.24.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/krb5kdc/oryx_127.8.24.1.keytab.
Entry for principal oryx/127.8.24.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/krb5kdc/oryx_127.8.24.1.keytab.
I20260808 16:52:34.308846 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.1:0
--local_ip_for_outbound_sockets=127.8.24.1
--webserver_interface=127.8.24.1
--webserver_port=0
--tserver_master_addrs=127.8.24.62:39641
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/krb5kdc/oryx_127.8.24.1.keytab
--principal=oryx/127.8.24.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:38953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:52:34.423359 20296 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:52:34.423612 20296 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:52:34.423645 20296 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:52:34.424813 20296 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:52:34.424854 20296 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:52:34.424875 20296 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 16:52:34.424890 20296 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.1
I20260808 16:52:34.426375 20296 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:38953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/krb5kdc/oryx_127.8.24.1.keytab
--principal=oryx/127.8.24.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.24.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.24.62:39641
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20296
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.1
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:52:34.426549 20296 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:52:34.426723 20296 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:52:34.431162 20306 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:34.431172 20307 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:34.431172 20312 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:34.431417 20296 server_base.cc:1034] running on GCE node
I20260808 16:52:34.431578 20296 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:52:34.431778 20296 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:52:34.432907 20296 hybrid_clock.cc:648] HybridClock initialized: now 1786207954432895 us; error 29 us; skew 500 ppm
Aug 08 16:52:34 dist-test-slave-6zbq krb5kdc[20112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207954, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.24.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:52:34.435294 20296 init.cc:377] Logged in from keytab as oryx/127.8.24.1@KRBTEST.COM (short username oryx)
I20260808 16:52:34.435685 20296 webserver.cc:460] Webserver started at http://127.8.24.1:35669/ using document root <none> and password file <none>
I20260808 16:52:34.435899 20296 fs_manager.cc:359] Metadata directory not provided
I20260808 16:52:34.435945 20296 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:52:34.436067 20296 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:52:34.437033 20296 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/minicluster-data/ts-0/data/instance:
uuid: "2771891c9a064fd6b6f4de95834948dc"
format_stamp: "Formatted at 2026-08-08 16:52:34 on dist-test-slave-6zbq"
server_key: "d323f258e7c96ab832de8dfd3fc87de0"
server_key_iv: "9164192334abea2b3540df017c66e908"
server_key_version: "encryptionkey@0"
I20260808 16:52:34.437361 20296 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/minicluster-data/ts-0/wal/instance:
uuid: "2771891c9a064fd6b6f4de95834948dc"
format_stamp: "Formatted at 2026-08-08 16:52:34 on dist-test-slave-6zbq"
server_key: "d323f258e7c96ab832de8dfd3fc87de0"
server_key_iv: "9164192334abea2b3540df017c66e908"
server_key_version: "encryptionkey@0"
I20260808 16:52:34.438573 20296 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 16:52:34.439196 20326 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:34.439368 20296 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 16:52:34.439441 20296 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/minicluster-data/ts-0/wal
uuid: "2771891c9a064fd6b6f4de95834948dc"
format_stamp: "Formatted at 2026-08-08 16:52:34 on dist-test-slave-6zbq"
server_key: "d323f258e7c96ab832de8dfd3fc87de0"
server_key_iv: "9164192334abea2b3540df017c66e908"
server_key_version: "encryptionkey@0"
I20260808 16:52:34.439500 20296 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:52:34.463269 20296 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:52:34.464840 20296 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:52:34.464962 20296 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:52:34.465404 20296 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:52:34.465442 20296 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:34.465471 20296 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:52:34.465497 20296 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 16:52:34 dist-test-slave-6zbq krb5kdc[20112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207954, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.24.1@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM
I20260808 16:52:34.471912 20296 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.1:45233
I20260808 16:52:34.471973 20519 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.1:45233 every 8 connection(s)
I20260808 16:52:34.472244 20296 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/minicluster-data/ts-0/data/info.pb
I20260808 16:52:34.474028 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 20296
I20260808 16:52:34.474133 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/minicluster-data/ts-0/wal/instance
I20260808 16:52:34.474327 8288 external_mini_cluster.cc:1428] Setting key f909d872cde3409218f4a7d715e257ca
I20260808 16:52:34.474714 20264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:34.465826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:52347 (local address 127.8.24.62:39641)
0808 16:52:34.465888 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:34.465890 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:34.466059 (+ 169us) server_negotiation.cc:407] Connection header received
0808 16:52:34.466536 (+ 477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:34.466538 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:34.466550 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:34.466624 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:34.467344 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:34.468078 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:34.469524 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:34.469838 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:34.469906 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:34.472063 (+ 2157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:34.472065 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:34.472065 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:34.472075 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:34.473388 (+ 1313us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:34.473713 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:34.473713 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:34.473714 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:34.473757 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:34.473948 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:34.473949 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:34.473949 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:34.474081 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:34.474146 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:34.474524 (+ 378us) server_negotiation.cc:299] Negotiation successful
0808 16:52:34.474621 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 16:52:34.474862 20346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:34.465779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:39641 (local address 127.8.24.1:52347)
0808 16:52:34.465943 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:34.465980 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:34.466448 (+ 468us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:34.466678 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:34.466683 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:34.466979 (+ 296us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:34.467240 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:34.467246 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:34.468190 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:34.468190 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:34.469414 (+ 1224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:34.469419 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:34.469923 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:34.469924 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:34.470058 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:34.470554 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:34.470562 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:34.471954 (+ 1392us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:52:34.473451 (+ 1497us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:34.473453 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:34.473455 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:34.473657 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:34.473799 (+ 142us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:34.473799 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:34.473800 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:34.473900 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:34.474178 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:52:34.474186 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:52:34.474381 (+ 195us) client_negotiation.cc:769] Sending connection context
0808 16:52:34.474497 (+ 116us) client_negotiation.cc:240] Negotiation successful
0808 16:52:34.474650 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":73,"threads_started":1}
I20260808 16:52:34.475266 20520 heartbeater.cc:347] Connected to a master server at 127.8.24.62:39641
I20260808 16:52:34.475358 20520 heartbeater.cc:464] Registering TS with master...
I20260808 16:52:34.475562 20520 heartbeater.cc:511] Master 127.8.24.62:39641 requested a full tablet report, sending...
I20260808 16:52:34.476109 20180 ts_manager.cc:194] Registered new tserver with Master: 2771891c9a064fd6b6f4de95834948dc (127.8.24.1:45233)
I20260808 16:52:34.476966 20180 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.24.1@KRBTEST.COM'} at 127.8.24.1:52347
I20260808 16:52:34.481289 20264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:34.476328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:37319 (local address 127.8.24.62:39641)
0808 16:52:34.476358 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:34.476359 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:34.476367 (+ 8us) server_negotiation.cc:407] Connection header received
0808 16:52:34.476423 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:34.476423 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:34.476432 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:34.476480 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:34.476792 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:34.477424 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:34.478075 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:34.478259 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:34.478309 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:34.479170 (+ 861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:34.479171 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:34.479171 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:34.479176 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:34.480365 (+ 1189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:34.480709 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:34.480710 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:34.480710 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:34.480749 (+ 39us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:34.480902 (+ 153us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:34.480902 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:34.480902 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:34.480978 (+ 76us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:34.481019 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:34.481182 (+ 163us) server_negotiation.cc:299] Negotiation successful
0808 16:52:34.481236 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260808 16:52:34.481364 20529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:34.476190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:39641 (local address 127.8.24.1:37319)
0808 16:52:34.476313 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:34.476324 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:34.476373 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:34.476484 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:34.476485 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:34.476653 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:34.476714 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:34.476715 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:34.477511 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:34.477511 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:34.478011 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:34.478013 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:34.478331 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:34.478331 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:34.478418 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:34.478752 (+ 334us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:34.478757 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:34.479085 (+ 328us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:52:34.480444 (+ 1359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:34.480445 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:34.480446 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:34.480644 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:34.480788 (+ 144us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:34.480788 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:34.480789 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:34.480846 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:34.481028 (+ 182us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:52:34.481029 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:52:34.481070 (+ 41us) client_negotiation.cc:769] Sending connection context
0808 16:52:34.481165 (+ 95us) client_negotiation.cc:240] Negotiation successful
0808 16:52:34.481254 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91,"mutex_wait_us":6,"thread_start_us":62,"threads_started":1}
WARNING: no policy specified for oryx/127.8.24.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.24.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.24.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/krb5kdc/oryx_127.8.24.2.keytab.
Entry for principal oryx/127.8.24.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/krb5kdc/oryx_127.8.24.2.keytab.
I20260808 16:52:34.497960 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.2:0
--local_ip_for_outbound_sockets=127.8.24.2
--webserver_interface=127.8.24.2
--webserver_port=0
--tserver_master_addrs=127.8.24.62:39641
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/krb5kdc/oryx_127.8.24.2.keytab
--principal=oryx/127.8.24.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:38953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:52:34.576647 20534 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:52:34.576828 20534 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:52:34.576848 20534 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:52:34.577937 20534 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:52:34.577975 20534 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:52:34.577996 20534 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 16:52:34.578024 20534 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.2
I20260808 16:52:34.579473 20534 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:38953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/krb5kdc/oryx_127.8.24.2.keytab
--principal=oryx/127.8.24.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.24.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.24.62:39641
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20534
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.2
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:52:34.579694 20534 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:52:34.579892 20534 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:52:34.584239 20543 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:34.584364 20540 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:34.584306 20541 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:34.584457 20534 server_base.cc:1034] running on GCE node
I20260808 16:52:34.584604 20534 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:52:34.584795 20534 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:52:34.585922 20534 hybrid_clock.cc:648] HybridClock initialized: now 1786207954585906 us; error 29 us; skew 500 ppm
Aug 08 16:52:34 dist-test-slave-6zbq krb5kdc[20112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207954, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.24.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:52:34.588092 20534 init.cc:377] Logged in from keytab as oryx/127.8.24.2@KRBTEST.COM (short username oryx)
I20260808 16:52:34.588461 20534 webserver.cc:460] Webserver started at http://127.8.24.2:35821/ using document root <none> and password file <none>
I20260808 16:52:34.588650 20534 fs_manager.cc:359] Metadata directory not provided
I20260808 16:52:34.588694 20534 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:52:34.588817 20534 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:52:34.589730 20534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/minicluster-data/ts-1/data/instance:
uuid: "7962ba67a970456781409e433168be76"
format_stamp: "Formatted at 2026-08-08 16:52:34 on dist-test-slave-6zbq"
server_key: "226590b79462fd306e10536d5e969102"
server_key_iv: "10a4833541abada15c4bef67f70ec038"
server_key_version: "encryptionkey@0"
I20260808 16:52:34.590034 20534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/minicluster-data/ts-1/wal/instance:
uuid: "7962ba67a970456781409e433168be76"
format_stamp: "Formatted at 2026-08-08 16:52:34 on dist-test-slave-6zbq"
server_key: "226590b79462fd306e10536d5e969102"
server_key_iv: "10a4833541abada15c4bef67f70ec038"
server_key_version: "encryptionkey@0"
I20260808 16:52:34.591279 20534 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 16:52:34.591957 20555 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:34.592097 20534 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 16:52:34.592177 20534 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/minicluster-data/ts-1/wal
uuid: "7962ba67a970456781409e433168be76"
format_stamp: "Formatted at 2026-08-08 16:52:34 on dist-test-slave-6zbq"
server_key: "226590b79462fd306e10536d5e969102"
server_key_iv: "10a4833541abada15c4bef67f70ec038"
server_key_version: "encryptionkey@0"
I20260808 16:52:34.592242 20534 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:52:34.609983 20534 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:52:34.611635 20534 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:52:34.611744 20534 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:52:34.612169 20534 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:52:34.612202 20534 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:34.612226 20534 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:52:34.612253 20534 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 16:52:34 dist-test-slave-6zbq krb5kdc[20112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207954, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.24.2@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM
I20260808 16:52:34.618526 20534 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.2:37649
I20260808 16:52:34.618587 20740 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.2:37649 every 8 connection(s)
I20260808 16:52:34.618911 20534 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/minicluster-data/ts-1/data/info.pb
I20260808 16:52:34.620666 20264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:34.612623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:46333 (local address 127.8.24.62:39641)
0808 16:52:34.612672 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:34.612674 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:34.612876 (+ 202us) server_negotiation.cc:407] Connection header received
0808 16:52:34.613304 (+ 428us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:34.613305 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:34.613319 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:34.613376 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:34.614100 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:34.614867 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:34.616010 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:34.616299 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:34.616373 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:34.618366 (+ 1993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:34.618368 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:34.618368 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:34.618376 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:34.619698 (+ 1322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:34.620024 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:34.620025 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:34.620026 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:34.620074 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:34.620229 (+ 155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:34.620229 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:34.620229 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:34.620317 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:34.620360 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:34.620542 (+ 182us) server_negotiation.cc:299] Negotiation successful
0808 16:52:34.620594 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 16:52:34.620813 20566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:34.612630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:39641 (local address 127.8.24.2:46333)
0808 16:52:34.612770 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:34.612804 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:34.613212 (+ 408us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:34.613440 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:34.613444 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:34.613745 (+ 301us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:34.613996 (+ 251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:34.614002 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:34.614963 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:34.614963 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:34.615945 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:34.615948 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:34.616400 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:34.616401 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:34.616531 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:34.616987 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:34.616995 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:34.618248 (+ 1253us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:52:34.619771 (+ 1523us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:34.619774 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:34.619775 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:34.619967 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:34.620114 (+ 147us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:34.620114 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:34.620115 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:34.620191 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:34.620361 (+ 170us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:52:34.620364 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:52:34.620459 (+ 95us) client_negotiation.cc:769] Sending connection context
0808 16:52:34.620529 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 16:52:34.620637 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":69,"threads_started":1}
I20260808 16:52:34.621217 20741 heartbeater.cc:347] Connected to a master server at 127.8.24.62:39641
I20260808 16:52:34.621294 20741 heartbeater.cc:464] Registering TS with master...
I20260808 16:52:34.621441 20741 heartbeater.cc:511] Master 127.8.24.62:39641 requested a full tablet report, sending...
I20260808 16:52:34.621773 20181 ts_manager.cc:194] Registered new tserver with Master: 7962ba67a970456781409e433168be76 (127.8.24.2:37649)
I20260808 16:52:34.621863 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 20534
I20260808 16:52:34.621941 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/minicluster-data/ts-1/wal/instance
I20260808 16:52:34.622118 8288 external_mini_cluster.cc:1428] Setting key 084fba9dbe48d71a443a794774bcbb28
I20260808 16:52:34.622222 20181 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.24.2@KRBTEST.COM'} at 127.8.24.2:46333
I20260808 16:52:34.626997 20264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:34.622141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:36507 (local address 127.8.24.62:39641)
0808 16:52:34.622172 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:34.622174 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:34.622375 (+ 201us) server_negotiation.cc:407] Connection header received
0808 16:52:34.622470 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:34.622471 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:34.622480 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:34.622536 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:34.622899 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:34.623386 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:34.624008 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:34.624203 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:34.624256 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:34.624971 (+ 715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:34.624972 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:34.624973 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:34.624979 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:34.626009 (+ 1030us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:34.626292 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:34.626294 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:34.626294 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:34.626363 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:34.626515 (+ 152us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:34.626516 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:34.626516 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:34.626657 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:34.626722 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:34.626893 (+ 171us) server_negotiation.cc:299] Negotiation successful
0808 16:52:34.626945 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260808 16:52:34.627066 20750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:34.622091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:39641 (local address 127.8.24.2:36507)
0808 16:52:34.622295 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:34.622304 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:34.622401 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:34.622550 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:34.622551 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:34.622743 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:34.622810 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:34.622811 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:34.623472 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:34.623473 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:34.623938 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:34.623939 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:34.624261 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:34.624262 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:34.624321 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:34.624661 (+ 340us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:34.624664 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:34.624897 (+ 233us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:52:34.626078 (+ 1181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:34.626078 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:34.626079 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:34.626233 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:34.626410 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:34.626411 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:34.626411 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:34.626464 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:34.626740 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:52:34.626741 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:52:34.626793 (+ 52us) client_negotiation.cc:769] Sending connection context
0808 16:52:34.626876 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 16:52:34.626965 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":79,"threads_started":1}
WARNING: no policy specified for oryx/127.8.24.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.24.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.24.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/krb5kdc/oryx_127.8.24.3.keytab.
Entry for principal oryx/127.8.24.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/krb5kdc/oryx_127.8.24.3.keytab.
I20260808 16:52:34.644608 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.3:0
--local_ip_for_outbound_sockets=127.8.24.3
--webserver_interface=127.8.24.3
--webserver_port=0
--tserver_master_addrs=127.8.24.62:39641
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/krb5kdc/oryx_127.8.24.3.keytab
--principal=oryx/127.8.24.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:38953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:52:34.722412 20756 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:52:34.722586 20756 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:52:34.722608 20756 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:52:34.723719 20756 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:52:34.723758 20756 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:52:34.723779 20756 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 16:52:34.723816 20756 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.3
I20260808 16:52:34.725244 20756 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:38953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/krb5kdc/oryx_127.8.24.3.keytab
--principal=oryx/127.8.24.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.24.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.24.62:39641
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20756
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.3
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:52:34.725452 20756 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:52:34.725642 20756 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:52:34.730278 20766 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:34.730330 20775 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:34.730321 20756 server_base.cc:1034] running on GCE node
W20260808 16:52:34.730261 20765 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:34.730667 20756 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:52:34.730875 20756 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:52:34.732007 20756 hybrid_clock.cc:648] HybridClock initialized: now 1786207954731995 us; error 28 us; skew 500 ppm
Aug 08 16:52:34 dist-test-slave-6zbq krb5kdc[20112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207954, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.24.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:52:34.734200 20756 init.cc:377] Logged in from keytab as oryx/127.8.24.3@KRBTEST.COM (short username oryx)
I20260808 16:52:34.734627 20756 webserver.cc:460] Webserver started at http://127.8.24.3:37473/ using document root <none> and password file <none>
I20260808 16:52:34.734812 20756 fs_manager.cc:359] Metadata directory not provided
I20260808 16:52:34.734858 20756 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:52:34.734975 20756 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:52:34.735903 20756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/minicluster-data/ts-2/data/instance:
uuid: "0e32c19034a34c04b2e5f339a9143b69"
format_stamp: "Formatted at 2026-08-08 16:52:34 on dist-test-slave-6zbq"
server_key: "762f9a7661d7c62059341a0b422168a3"
server_key_iv: "b48875202b0b01f537f53e268c5c0536"
server_key_version: "encryptionkey@0"
I20260808 16:52:34.736217 20756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/minicluster-data/ts-2/wal/instance:
uuid: "0e32c19034a34c04b2e5f339a9143b69"
format_stamp: "Formatted at 2026-08-08 16:52:34 on dist-test-slave-6zbq"
server_key: "762f9a7661d7c62059341a0b422168a3"
server_key_iv: "b48875202b0b01f537f53e268c5c0536"
server_key_version: "encryptionkey@0"
I20260808 16:52:34.737367 20756 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 16:52:34.737977 20785 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:34.738131 20756 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 16:52:34.738214 20756 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/minicluster-data/ts-2/wal
uuid: "0e32c19034a34c04b2e5f339a9143b69"
format_stamp: "Formatted at 2026-08-08 16:52:34 on dist-test-slave-6zbq"
server_key: "762f9a7661d7c62059341a0b422168a3"
server_key_iv: "b48875202b0b01f537f53e268c5c0536"
server_key_version: "encryptionkey@0"
I20260808 16:52:34.738279 20756 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:52:34.748193 20756 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:52:34.749680 20756 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:52:34.749799 20756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:52:34.750212 20756 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:52:34.750252 20756 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:34.750278 20756 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:52:34.750311 20756 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 16:52:34 dist-test-slave-6zbq krb5kdc[20112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207954, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.24.3@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM
I20260808 16:52:34.756587 20756 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.3:34921
I20260808 16:52:34.756701 20981 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.3:34921 every 8 connection(s)
I20260808 16:52:34.757005 20756 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/minicluster-data/ts-2/data/info.pb
I20260808 16:52:34.758509 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 20756
I20260808 16:52:34.758627 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/minicluster-data/ts-2/wal/instance
I20260808 16:52:34.758834 8288 external_mini_cluster.cc:1428] Setting key 5c05b05c4bfdec0a731e3021680b4289
I20260808 16:52:34.759763 20264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:34.750648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:50487 (local address 127.8.24.62:39641)
0808 16:52:34.750713 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:34.750716 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:34.750840 (+ 124us) server_negotiation.cc:407] Connection header received
0808 16:52:34.751254 (+ 414us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:34.751255 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:34.751268 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:34.751332 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:34.751891 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:34.752661 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:34.754061 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:34.754364 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:34.754443 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:34.756714 (+ 2271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:34.756715 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:34.756716 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:34.756723 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:34.758290 (+ 1567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:34.758790 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:34.758792 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:34.758792 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:34.758864 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:34.759128 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:34.759128 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:34.759129 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:34.759260 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:34.759319 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:34.759593 (+ 274us) server_negotiation.cc:299] Negotiation successful
0808 16:52:34.759675 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 16:52:34.759898 20795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:34.750593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:39641 (local address 127.8.24.3:50487)
0808 16:52:34.750732 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:34.750768 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:34.751174 (+ 406us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:34.751390 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:34.751392 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:34.751610 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:34.751821 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:34.751826 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:34.752746 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:34.752747 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:34.753970 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:34.753973 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:34.754461 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:34.754462 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:34.754580 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:34.755012 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:34.755028 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:34.756613 (+ 1585us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:52:34.758398 (+ 1785us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:34.758404 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:34.758407 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:34.758699 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:34.758928 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:34.758928 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:34.758929 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:34.759055 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:34.759319 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:52:34.759324 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:52:34.759467 (+ 143us) client_negotiation.cc:769] Sending connection context
0808 16:52:34.759553 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 16:52:34.759709 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":81,"threads_started":1}
I20260808 16:52:34.760258 20982 heartbeater.cc:347] Connected to a master server at 127.8.24.62:39641
I20260808 16:52:34.760329 20982 heartbeater.cc:464] Registering TS with master...
I20260808 16:52:34.760483 20982 heartbeater.cc:511] Master 127.8.24.62:39641 requested a full tablet report, sending...
I20260808 16:52:34.760850 20180 ts_manager.cc:194] Registered new tserver with Master: 0e32c19034a34c04b2e5f339a9143b69 (127.8.24.3:34921)
I20260808 16:52:34.761260 20180 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.24.3@KRBTEST.COM'} at 127.8.24.3:50487
I20260808 16:52:34.765760 20264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:34.761206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:41899 (local address 127.8.24.62:39641)
0808 16:52:34.761241 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:34.761243 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:34.761293 (+ 50us) server_negotiation.cc:407] Connection header received
0808 16:52:34.761380 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:34.761381 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:34.761390 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:34.761441 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:34.761766 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:34.762275 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:34.762921 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:34.763088 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:34.763137 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:34.763776 (+ 639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:34.763777 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:34.763777 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:34.763783 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:34.764939 (+ 1156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:34.765169 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:34.765170 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:34.765170 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:34.765209 (+ 39us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:34.765312 (+ 103us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:34.765313 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:34.765313 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:34.765404 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:34.765453 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:34.765617 (+ 164us) server_negotiation.cc:299] Negotiation successful
0808 16:52:34.765678 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 16:52:34.765771 20989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:34.761131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:39641 (local address 127.8.24.3:41899)
0808 16:52:34.761237 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:34.761247 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:34.761307 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:34.761453 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:34.761453 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:34.761636 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:34.761697 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:34.761698 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:34.762367 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:34.762368 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:34.762865 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:34.762866 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:34.763142 (+ 276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:34.763142 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:34.763183 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:34.763470 (+ 287us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:34.763474 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:34.763713 (+ 239us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:52:34.764995 (+ 1282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:34.764996 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:34.764996 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:34.765128 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:34.765241 (+ 113us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:34.765242 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:34.765242 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:34.765282 (+ 40us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:34.765472 (+ 190us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:52:34.765473 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:52:34.765517 (+ 44us) client_negotiation.cc:769] Sending connection context
0808 16:52:34.765577 (+ 60us) client_negotiation.cc:240] Negotiation successful
0808 16:52:34.765672 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":82,"thread_start_us":57,"threads_started":1}
I20260808 16:52:34.770375 8288 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 16:52:34 dist-test-slave-6zbq krb5kdc[20112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207954, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 16:52:34 dist-test-slave-6zbq krb5kdc[20112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:52:34 dist-test-slave-6zbq krb5kdc[20112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
W20260808 16:52:34.790251 20264 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:52:34.785651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55396 (local address 127.8.24.62:39641)
0808 16:52:34.785696 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:34.785698 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:34.785708 (+ 10us) server_negotiation.cc:407] Connection header received
0808 16:52:34.785763 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:34.785764 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:34.785772 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:34.785819 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:34.786304 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:34.786773 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:34.787431 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:34.787595 (+ 164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:34.787635 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:34.790176 (+ 2541us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55396: BlockingRecv error: recv got EOF from 127.0.0.1:55396 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
Aug 08 16:52:34 dist-test-slave-6zbq krb5kdc[20112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207954, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM
I20260808 16:52:34.796828 20264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:34.790958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55404 (local address 127.8.24.62:39641)
0808 16:52:34.791005 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:34.791006 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:34.791049 (+ 43us) server_negotiation.cc:407] Connection header received
0808 16:52:34.791122 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:34.791122 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:34.791131 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:34.791174 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:34.791587 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:34.792015 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:34.792634 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:34.792813 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:34.792864 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:34.794704 (+ 1840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:34.794706 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:34.794706 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:34.794713 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:34.795915 (+ 1202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:34.796200 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:34.796201 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:34.796202 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:34.796251 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:34.796407 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:34.796408 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:34.796408 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:34.796499 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:34.796579 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:34.796681 (+ 102us) server_negotiation.cc:299] Negotiation successful
0808 16:52:34.796749 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 16:52:34.798273 20180 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55404:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 16:52:34.799782 20180 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 16:52:34.805727 21026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:34.802023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:45233 (local address 127.0.0.1:49408)
0808 16:52:34.802253 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:34.802262 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:34.802315 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:34.802555 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:34.802556 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:34.802558 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:52:34.802596 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:34.802597 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:34.803493 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:34.803494 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:34.804620 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:34.804623 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:34.805440 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:34.805441 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:34.805528 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:34.805542 (+ 14us) client_negotiation.cc:769] Sending connection context
0808 16:52:34.805606 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 16:52:34.805657 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":75,"threads_started":1}
I20260808 16:52:34.805845 21025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:34.801923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41986 (local address 127.8.24.3:34921)
0808 16:52:34.802454 (+ 531us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:34.802459 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:34.802467 (+ 8us) server_negotiation.cc:407] Connection header received
0808 16:52:34.802500 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:34.802502 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:34.802515 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:34.802616 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:52:34.802752 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:34.803828 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:34.804756 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:34.805514 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:34.805586 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:34.805655 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:34.805712 (+ 57us) server_negotiation.cc:299] Negotiation successful
0808 16:52:34.805745 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":496,"thread_start_us":87,"threads_started":1}
I20260808 16:52:34.805895 21023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:34.801838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.3:34921 (local address 127.0.0.1:41986)
0808 16:52:34.801966 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:34.802007 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:34.802071 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:34.802614 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:34.802615 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:34.802617 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:52:34.802645 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:34.802646 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:34.803922 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:34.803923 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:34.804670 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:34.804673 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:34.805585 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:34.805586 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:34.805648 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:34.805650 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 16:52:34.805699 (+ 49us) client_negotiation.cc:240] Negotiation successful
0808 16:52:34.805730 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":53,"threads_started":1}
I20260808 16:52:34.806186 21029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:34.802161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49408 (local address 127.8.24.1:45233)
0808 16:52:34.802328 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:34.802331 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:34.802355 (+ 24us) server_negotiation.cc:407] Connection header received
0808 16:52:34.802395 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:34.802397 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:34.802412 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:34.802491 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:52:34.802674 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:34.803416 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:34.804714 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:34.805364 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:34.805423 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:34.805499 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:34.806038 (+ 539us) server_negotiation.cc:299] Negotiation successful
0808 16:52:34.806083 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":73,"threads_started":1}
I20260808 16:52:34.806309 21020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:34.801838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.2:37649 (local address 127.0.0.1:48898)
0808 16:52:34.802146 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:34.802155 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:34.802205 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:34.802427 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:34.802429 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:34.802432 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:52:34.802500 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:34.802502 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:34.803872 (+ 1370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:34.803873 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:34.804907 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:34.804910 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:34.806072 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:34.806073 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:34.806142 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:34.806146 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 16:52:34.806202 (+ 56us) client_negotiation.cc:240] Negotiation successful
0808 16:52:34.806239 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"mutex_wait_us":4,"thread_start_us":71,"threads_started":1}
I20260808 16:52:34.806571 20873 tablet_service.cc:1467] Processing CreateTablet for tablet 3aeac13b872e400ea727f0b888b48422 (DEFAULT_TABLE table=test-table [id=5b6bb7916bcf4c07bec5b3b399b272fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:52:34.806687 21027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:34.802101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48898 (local address 127.8.24.2:37649)
0808 16:52:34.802228 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:34.802232 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:34.802242 (+ 10us) server_negotiation.cc:407] Connection header received
0808 16:52:34.802281 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:34.802284 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:34.802299 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:34.802415 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:52:34.802569 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:34.803778 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:34.804990 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:34.805769 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:34.806057 (+ 288us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:34.806143 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:34.806217 (+ 74us) server_negotiation.cc:299] Negotiation successful
0808 16:52:34.806595 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97,"thread_start_us":68,"threads_started":1}
I20260808 16:52:34.806901 20873 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3aeac13b872e400ea727f0b888b48422. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:52:34.806841 20413 tablet_service.cc:1467] Processing CreateTablet for tablet 3aeac13b872e400ea727f0b888b48422 (DEFAULT_TABLE table=test-table [id=5b6bb7916bcf4c07bec5b3b399b272fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:52:34.807171 20413 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3aeac13b872e400ea727f0b888b48422. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:52:34.807356 20634 tablet_service.cc:1467] Processing CreateTablet for tablet 3aeac13b872e400ea727f0b888b48422 (DEFAULT_TABLE table=test-table [id=5b6bb7916bcf4c07bec5b3b399b272fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:52:34.807683 20634 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3aeac13b872e400ea727f0b888b48422. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:52:34.810148 21030 tablet_bootstrap.cc:492] T 3aeac13b872e400ea727f0b888b48422 P 2771891c9a064fd6b6f4de95834948dc: Bootstrap starting.
I20260808 16:52:34.810485 21031 tablet_bootstrap.cc:492] T 3aeac13b872e400ea727f0b888b48422 P 0e32c19034a34c04b2e5f339a9143b69: Bootstrap starting.
I20260808 16:52:34.810909 21030 tablet_bootstrap.cc:654] T 3aeac13b872e400ea727f0b888b48422 P 2771891c9a064fd6b6f4de95834948dc: Neither blocks nor log segments found. Creating new log.
I20260808 16:52:34.811048 21031 tablet_bootstrap.cc:654] T 3aeac13b872e400ea727f0b888b48422 P 0e32c19034a34c04b2e5f339a9143b69: Neither blocks nor log segments found. Creating new log.
I20260808 16:52:34.811206 21032 tablet_bootstrap.cc:492] T 3aeac13b872e400ea727f0b888b48422 P 7962ba67a970456781409e433168be76: Bootstrap starting.
I20260808 16:52:34.811233 21030 log.cc:824] T 3aeac13b872e400ea727f0b888b48422 P 2771891c9a064fd6b6f4de95834948dc: Log is configured to *not* fsync() on all Append() calls
I20260808 16:52:34.811344 21031 log.cc:824] T 3aeac13b872e400ea727f0b888b48422 P 0e32c19034a34c04b2e5f339a9143b69: Log is configured to *not* fsync() on all Append() calls
I20260808 16:52:34.811832 21032 tablet_bootstrap.cc:654] T 3aeac13b872e400ea727f0b888b48422 P 7962ba67a970456781409e433168be76: Neither blocks nor log segments found. Creating new log.
I20260808 16:52:34.811964 21030 tablet_bootstrap.cc:492] T 3aeac13b872e400ea727f0b888b48422 P 2771891c9a064fd6b6f4de95834948dc: No bootstrap required, opened a new log
I20260808 16:52:34.812036 21030 ts_tablet_manager.cc:1397] T 3aeac13b872e400ea727f0b888b48422 P 2771891c9a064fd6b6f4de95834948dc: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 16:52:34.812063 21031 tablet_bootstrap.cc:492] T 3aeac13b872e400ea727f0b888b48422 P 0e32c19034a34c04b2e5f339a9143b69: No bootstrap required, opened a new log
I20260808 16:52:34.812122 21031 ts_tablet_manager.cc:1397] T 3aeac13b872e400ea727f0b888b48422 P 0e32c19034a34c04b2e5f339a9143b69: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 16:52:34.812139 21032 log.cc:824] T 3aeac13b872e400ea727f0b888b48422 P 7962ba67a970456781409e433168be76: Log is configured to *not* fsync() on all Append() calls
I20260808 16:52:34.812743 21032 tablet_bootstrap.cc:492] T 3aeac13b872e400ea727f0b888b48422 P 7962ba67a970456781409e433168be76: No bootstrap required, opened a new log
I20260808 16:52:34.812800 21032 ts_tablet_manager.cc:1397] T 3aeac13b872e400ea727f0b888b48422 P 7962ba67a970456781409e433168be76: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 16:52:34.813931 21030 raft_consensus.cc:348] T 3aeac13b872e400ea727f0b888b48422 P 2771891c9a064fd6b6f4de95834948dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e32c19034a34c04b2e5f339a9143b69" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 34921 } } peers { permanent_uuid: "2771891c9a064fd6b6f4de95834948dc" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 45233 } } peers { permanent_uuid: "7962ba67a970456781409e433168be76" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 37649 } }
I20260808 16:52:34.814097 21030 raft_consensus.cc:374] T 3aeac13b872e400ea727f0b888b48422 P 2771891c9a064fd6b6f4de95834948dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:52:34.814138 21030 raft_consensus.cc:729] T 3aeac13b872e400ea727f0b888b48422 P 2771891c9a064fd6b6f4de95834948dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2771891c9a064fd6b6f4de95834948dc, State: Initialized, Role: FOLLOWER
I20260808 16:52:34.814110 21031 raft_consensus.cc:348] T 3aeac13b872e400ea727f0b888b48422 P 0e32c19034a34c04b2e5f339a9143b69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e32c19034a34c04b2e5f339a9143b69" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 34921 } } peers { permanent_uuid: "2771891c9a064fd6b6f4de95834948dc" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 45233 } } peers { permanent_uuid: "7962ba67a970456781409e433168be76" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 37649 } }
I20260808 16:52:34.814247 21031 raft_consensus.cc:374] T 3aeac13b872e400ea727f0b888b48422 P 0e32c19034a34c04b2e5f339a9143b69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:52:34.814247 21030 consensus_queue.cc:260] T 3aeac13b872e400ea727f0b888b48422 P 2771891c9a064fd6b6f4de95834948dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e32c19034a34c04b2e5f339a9143b69" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 34921 } } peers { permanent_uuid: "2771891c9a064fd6b6f4de95834948dc" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 45233 } } peers { permanent_uuid: "7962ba67a970456781409e433168be76" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 37649 } }
I20260808 16:52:34.814285 21031 raft_consensus.cc:729] T 3aeac13b872e400ea727f0b888b48422 P 0e32c19034a34c04b2e5f339a9143b69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e32c19034a34c04b2e5f339a9143b69, State: Initialized, Role: FOLLOWER
I20260808 16:52:34.814286 21032 raft_consensus.cc:348] T 3aeac13b872e400ea727f0b888b48422 P 7962ba67a970456781409e433168be76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e32c19034a34c04b2e5f339a9143b69" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 34921 } } peers { permanent_uuid: "2771891c9a064fd6b6f4de95834948dc" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 45233 } } peers { permanent_uuid: "7962ba67a970456781409e433168be76" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 37649 } }
I20260808 16:52:34.814435 21031 consensus_queue.cc:260] T 3aeac13b872e400ea727f0b888b48422 P 0e32c19034a34c04b2e5f339a9143b69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e32c19034a34c04b2e5f339a9143b69" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 34921 } } peers { permanent_uuid: "2771891c9a064fd6b6f4de95834948dc" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 45233 } } peers { permanent_uuid: "7962ba67a970456781409e433168be76" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 37649 } }
I20260808 16:52:34.814502 21032 raft_consensus.cc:374] T 3aeac13b872e400ea727f0b888b48422 P 7962ba67a970456781409e433168be76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:52:34.814522 21030 ts_tablet_manager.cc:1428] T 3aeac13b872e400ea727f0b888b48422 P 2771891c9a064fd6b6f4de95834948dc: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 16:52:34.814546 21032 raft_consensus.cc:729] T 3aeac13b872e400ea727f0b888b48422 P 7962ba67a970456781409e433168be76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7962ba67a970456781409e433168be76, State: Initialized, Role: FOLLOWER
I20260808 16:52:34.814656 20520 heartbeater.cc:503] Master 127.8.24.62:39641 was elected leader, sending a full tablet report...
I20260808 16:52:34.814666 21031 ts_tablet_manager.cc:1428] T 3aeac13b872e400ea727f0b888b48422 P 0e32c19034a34c04b2e5f339a9143b69: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 16:52:34.814668 21032 consensus_queue.cc:260] T 3aeac13b872e400ea727f0b888b48422 P 7962ba67a970456781409e433168be76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e32c19034a34c04b2e5f339a9143b69" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 34921 } } peers { permanent_uuid: "2771891c9a064fd6b6f4de95834948dc" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 45233 } } peers { permanent_uuid: "7962ba67a970456781409e433168be76" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 37649 } }
I20260808 16:52:34.814785 20982 heartbeater.cc:503] Master 127.8.24.62:39641 was elected leader, sending a full tablet report...
I20260808 16:52:34.814929 21032 ts_tablet_manager.cc:1428] T 3aeac13b872e400ea727f0b888b48422 P 7962ba67a970456781409e433168be76: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 16:52:34.814947 20741 heartbeater.cc:503] Master 127.8.24.62:39641 was elected leader, sending a full tablet report...
W20260808 16:52:34.869997 20742 tablet.cc:2369] T 3aeac13b872e400ea727f0b888b48422 P 7962ba67a970456781409e433168be76: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 16:52:34.973033 20521 tablet.cc:2369] T 3aeac13b872e400ea727f0b888b48422 P 2771891c9a064fd6b6f4de95834948dc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 16:52:35.007700 20983 tablet.cc:2369] T 3aeac13b872e400ea727f0b888b48422 P 0e32c19034a34c04b2e5f339a9143b69: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 16:52:35.217640 21038 raft_consensus.cc:482] T 3aeac13b872e400ea727f0b888b48422 P 0e32c19034a34c04b2e5f339a9143b69 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 16:52:35.217782 21038 raft_consensus.cc:504] T 3aeac13b872e400ea727f0b888b48422 P 0e32c19034a34c04b2e5f339a9143b69 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e32c19034a34c04b2e5f339a9143b69" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 34921 } } peers { permanent_uuid: "2771891c9a064fd6b6f4de95834948dc" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 45233 } } peers { permanent_uuid: "7962ba67a970456781409e433168be76" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 37649 } }
I20260808 16:52:35.218098 21038 leader_election.cc:288] T 3aeac13b872e400ea727f0b888b48422 P 0e32c19034a34c04b2e5f339a9143b69 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2771891c9a064fd6b6f4de95834948dc (127.8.24.1:45233), 7962ba67a970456781409e433168be76 (127.8.24.2:37649)
I20260808 16:52:35.220870 21029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:35.218393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:47195 (local address 127.8.24.1:45233)
0808 16:52:35.218462 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:35.218464 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:35.218475 (+ 11us) server_negotiation.cc:407] Connection header received
0808 16:52:35.218506 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:35.218507 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:35.218520 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:35.218576 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:52:35.218747 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:35.219229 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:35.220207 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:35.220564 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:35.220616 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:35.220626 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:35.220765 (+ 139us) server_negotiation.cc:299] Negotiation successful
0808 16:52:35.220801 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 16:52:35.220880 21027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:35.218399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:37553 (local address 127.8.24.2:37649)
0808 16:52:35.218446 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:35.218449 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:35.218555 (+ 106us) server_negotiation.cc:407] Connection header received
0808 16:52:35.218608 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:35.218609 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:35.218621 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:35.218676 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:52:35.218827 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:35.219259 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:35.220232 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:35.220582 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:35.220653 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:35.220663 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:35.220786 (+ 123us) server_negotiation.cc:299] Negotiation successful
0808 16:52:35.220817 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 16:52:35.220894 21041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:35.218301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.2:37649 (local address 127.8.24.3:37553)
0808 16:52:35.218488 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:35.218497 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:35.218557 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:35.218685 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:35.218686 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:35.218687 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:52:35.218767 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:35.218768 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:35.219363 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:35.219364 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:35.220121 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:35.220123 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:35.220664 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:35.220665 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:35.220723 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:35.220726 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 16:52:35.220775 (+ 49us) client_negotiation.cc:240] Negotiation successful
0808 16:52:35.220810 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"mutex_wait_us":2,"thread_start_us":113,"threads_started":1}
I20260808 16:52:35.220896 20795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:35.218279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:45233 (local address 127.8.24.3:47195)
0808 16:52:35.218330 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:35.218361 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:35.218410 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:35.218588 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:35.218589 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:35.218590 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:52:35.218681 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:35.218683 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:35.219295 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:35.219296 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:35.220118 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:35.220121 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:35.220633 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:35.220634 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:35.220701 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:35.220705 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 16:52:35.220745 (+ 40us) client_negotiation.cc:240] Negotiation successful
0808 16:52:35.220785 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30,"mutex_wait_us":59}
I20260808 16:52:35.221226 20664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3aeac13b872e400ea727f0b888b48422" candidate_uuid: "0e32c19034a34c04b2e5f339a9143b69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7962ba67a970456781409e433168be76" is_pre_election: true
I20260808 16:52:35.221374 20664 raft_consensus.cc:2454] T 3aeac13b872e400ea727f0b888b48422 P 7962ba67a970456781409e433168be76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e32c19034a34c04b2e5f339a9143b69 in term 0.
I20260808 16:52:35.221412 20433 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3aeac13b872e400ea727f0b888b48422" candidate_uuid: "0e32c19034a34c04b2e5f339a9143b69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2771891c9a064fd6b6f4de95834948dc" is_pre_election: true
I20260808 16:52:35.221557 20433 raft_consensus.cc:2454] T 3aeac13b872e400ea727f0b888b48422 P 2771891c9a064fd6b6f4de95834948dc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e32c19034a34c04b2e5f339a9143b69 in term 0.
I20260808 16:52:35.221572 20787 leader_election.cc:302] T 3aeac13b872e400ea727f0b888b48422 P 0e32c19034a34c04b2e5f339a9143b69 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e32c19034a34c04b2e5f339a9143b69, 7962ba67a970456781409e433168be76; no voters:
I20260808 16:52:35.221692 21038 raft_consensus.cc:2781] T 3aeac13b872e400ea727f0b888b48422 P 0e32c19034a34c04b2e5f339a9143b69 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 16:52:35.221763 21038 raft_consensus.cc:482] T 3aeac13b872e400ea727f0b888b48422 P 0e32c19034a34c04b2e5f339a9143b69 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 16:52:35.221794 21038 raft_consensus.cc:3037] T 3aeac13b872e400ea727f0b888b48422 P 0e32c19034a34c04b2e5f339a9143b69 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:52:35.222797 21038 raft_consensus.cc:504] T 3aeac13b872e400ea727f0b888b48422 P 0e32c19034a34c04b2e5f339a9143b69 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e32c19034a34c04b2e5f339a9143b69" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 34921 } } peers { permanent_uuid: "2771891c9a064fd6b6f4de95834948dc" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 45233 } } peers { permanent_uuid: "7962ba67a970456781409e433168be76" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 37649 } }
I20260808 16:52:35.222906 21038 leader_election.cc:288] T 3aeac13b872e400ea727f0b888b48422 P 0e32c19034a34c04b2e5f339a9143b69 [CANDIDATE]: Term 1 election: Requested vote from peers 2771891c9a064fd6b6f4de95834948dc (127.8.24.1:45233), 7962ba67a970456781409e433168be76 (127.8.24.2:37649)
I20260808 16:52:35.223050 20664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3aeac13b872e400ea727f0b888b48422" candidate_uuid: "0e32c19034a34c04b2e5f339a9143b69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7962ba67a970456781409e433168be76"
I20260808 16:52:35.223124 20664 raft_consensus.cc:3037] T 3aeac13b872e400ea727f0b888b48422 P 7962ba67a970456781409e433168be76 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:52:35.223115 20433 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3aeac13b872e400ea727f0b888b48422" candidate_uuid: "0e32c19034a34c04b2e5f339a9143b69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2771891c9a064fd6b6f4de95834948dc"
I20260808 16:52:35.223193 20433 raft_consensus.cc:3037] T 3aeac13b872e400ea727f0b888b48422 P 2771891c9a064fd6b6f4de95834948dc [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:52:35.223803 20664 raft_consensus.cc:2454] T 3aeac13b872e400ea727f0b888b48422 P 7962ba67a970456781409e433168be76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e32c19034a34c04b2e5f339a9143b69 in term 1.
I20260808 16:52:35.223860 20433 raft_consensus.cc:2454] T 3aeac13b872e400ea727f0b888b48422 P 2771891c9a064fd6b6f4de95834948dc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e32c19034a34c04b2e5f339a9143b69 in term 1.
I20260808 16:52:35.223981 20787 leader_election.cc:302] T 3aeac13b872e400ea727f0b888b48422 P 0e32c19034a34c04b2e5f339a9143b69 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e32c19034a34c04b2e5f339a9143b69, 7962ba67a970456781409e433168be76; no voters:
I20260808 16:52:35.224073 21038 raft_consensus.cc:2781] T 3aeac13b872e400ea727f0b888b48422 P 0e32c19034a34c04b2e5f339a9143b69 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:52:35.224226 21038 raft_consensus.cc:686] T 3aeac13b872e400ea727f0b888b48422 P 0e32c19034a34c04b2e5f339a9143b69 [term 1 LEADER]: Becoming Leader. State: Replica: 0e32c19034a34c04b2e5f339a9143b69, State: Running, Role: LEADER
I20260808 16:52:35.224303 21038 consensus_queue.cc:237] T 3aeac13b872e400ea727f0b888b48422 P 0e32c19034a34c04b2e5f339a9143b69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e32c19034a34c04b2e5f339a9143b69" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 34921 } } peers { permanent_uuid: "2771891c9a064fd6b6f4de95834948dc" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 45233 } } peers { permanent_uuid: "7962ba67a970456781409e433168be76" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 37649 } }
I20260808 16:52:35.225023 20179 catalog_manager.cc:5495] T 3aeac13b872e400ea727f0b888b48422 P 0e32c19034a34c04b2e5f339a9143b69 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0e32c19034a34c04b2e5f339a9143b69 (127.8.24.3). New cstate: current_term: 1 leader_uuid: "0e32c19034a34c04b2e5f339a9143b69" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e32c19034a34c04b2e5f339a9143b69" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 34921 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2771891c9a064fd6b6f4de95834948dc" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 45233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7962ba67a970456781409e433168be76" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 37649 } health_report { overall_health: UNKNOWN } } }
I20260808 16:52:35.244325 21025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:35.242270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41990 (local address 127.8.24.3:34921)
0808 16:52:35.242308 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:35.242310 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:35.242318 (+ 8us) server_negotiation.cc:407] Connection header received
0808 16:52:35.242407 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:35.242408 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:35.242441 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:35.242496 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 16:52:35.242591 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:35.242991 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:35.243686 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:35.243850 (+ 164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:35.243890 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:35.243997 (+ 107us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 16:52:35.244076 (+ 79us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 16:52:35.244132 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:35.244272 (+ 140us) server_negotiation.cc:299] Negotiation successful
0808 16:52:35.244284 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 16:52:35.245740 20664 raft_consensus.cc:1261] T 3aeac13b872e400ea727f0b888b48422 P 7962ba67a970456781409e433168be76 [term 1 FOLLOWER]: Refusing update from remote peer 0e32c19034a34c04b2e5f339a9143b69: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:52:35.245756 20433 raft_consensus.cc:1261] T 3aeac13b872e400ea727f0b888b48422 P 2771891c9a064fd6b6f4de95834948dc [term 1 FOLLOWER]: Refusing update from remote peer 0e32c19034a34c04b2e5f339a9143b69: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:52:35.246002 21038 consensus_queue.cc:1035] T 3aeac13b872e400ea727f0b888b48422 P 0e32c19034a34c04b2e5f339a9143b69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7962ba67a970456781409e433168be76" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 37649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:52:35.246129 21042 consensus_queue.cc:1035] T 3aeac13b872e400ea727f0b888b48422 P 0e32c19034a34c04b2e5f339a9143b69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2771891c9a064fd6b6f4de95834948dc" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:52:35.248075 21046 mvcc.cc:205] Tried to move back new op lower bound from 7316307784684576768 to 7316307784599162880. Current Snapshot: MvccSnapshot[applied={T|T < 7316307784684576768}]
I20260808 16:52:35.248761 21051 mvcc.cc:205] Tried to move back new op lower bound from 7316307784684576768 to 7316307784599162880. Current Snapshot: MvccSnapshot[applied={T|T < 7316307784684576768}]
I20260808 16:52:35.250710 20179 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55404:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 16:52:35.250792 20179 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55404:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 16:52:35.251415 20179 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e63efa3dec1242f980a4d3123bd3f739: Sending DeleteTablet for 3 replicas of tablet 3aeac13b872e400ea727f0b888b48422
I20260808 16:52:35.251726 20873 tablet_service.cc:1514] Processing DeleteTablet for tablet 3aeac13b872e400ea727f0b888b48422 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:52:35 UTC) from {username='oryx', principal='oryx/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:41986
I20260808 16:52:35.251763 20634 tablet_service.cc:1514] Processing DeleteTablet for tablet 3aeac13b872e400ea727f0b888b48422 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:52:35 UTC) from {username='oryx', principal='oryx/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:48898
I20260808 16:52:35.251765 20413 tablet_service.cc:1514] Processing DeleteTablet for tablet 3aeac13b872e400ea727f0b888b48422 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:52:35 UTC) from {username='oryx', principal='oryx/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:49408
I20260808 16:52:35.251915 21062 tablet_replica.cc:330] T 3aeac13b872e400ea727f0b888b48422 P 7962ba67a970456781409e433168be76: stopping tablet replica
I20260808 16:52:35.252012 21062 raft_consensus.cc:2229] T 3aeac13b872e400ea727f0b888b48422 P 7962ba67a970456781409e433168be76 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 16:52:35.252116 20179 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55404:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 16:52:35.252250 20179 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 16:52:35.252424 21062 raft_consensus.cc:2258] T 3aeac13b872e400ea727f0b888b48422 P 7962ba67a970456781409e433168be76 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:52:35.252974 21062 ts_tablet_manager.cc:1905] T 3aeac13b872e400ea727f0b888b48422 P 7962ba67a970456781409e433168be76: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:52:35.253360 20634 tablet_service.cc:1467] Processing CreateTablet for tablet 0c1801d6f33c463cab1a05fcf8c824b9 (DEFAULT_TABLE table=test-table [id=4f5597de33ce4d7ab5c5539abee900af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:52:35.253401 20873 tablet_service.cc:1467] Processing CreateTablet for tablet 0c1801d6f33c463cab1a05fcf8c824b9 (DEFAULT_TABLE table=test-table [id=4f5597de33ce4d7ab5c5539abee900af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:52:35.253472 20634 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c1801d6f33c463cab1a05fcf8c824b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:52:35.253484 20873 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c1801d6f33c463cab1a05fcf8c824b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:52:35.254617 21062 ts_tablet_manager.cc:1918] T 3aeac13b872e400ea727f0b888b48422 P 7962ba67a970456781409e433168be76: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 16:52:35.254678 21062 log.cc:1196] T 3aeac13b872e400ea727f0b888b48422 P 7962ba67a970456781409e433168be76: Deleting WAL directory at /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/minicluster-data/ts-1/wal/wals/3aeac13b872e400ea727f0b888b48422
I20260808 16:52:35.254691 21031 tablet_bootstrap.cc:492] T 0c1801d6f33c463cab1a05fcf8c824b9 P 0e32c19034a34c04b2e5f339a9143b69: Bootstrap starting.
I20260808 16:52:35.254709 21032 tablet_bootstrap.cc:492] T 0c1801d6f33c463cab1a05fcf8c824b9 P 7962ba67a970456781409e433168be76: Bootstrap starting.
I20260808 16:52:35.254925 21062 ts_tablet_manager.cc:1939] T 3aeac13b872e400ea727f0b888b48422 P 7962ba67a970456781409e433168be76: Deleting consensus metadata
I20260808 16:52:35.255064 21032 tablet_bootstrap.cc:654] T 0c1801d6f33c463cab1a05fcf8c824b9 P 7962ba67a970456781409e433168be76: Neither blocks nor log segments found. Creating new log.
I20260808 16:52:35.255071 21031 tablet_bootstrap.cc:654] T 0c1801d6f33c463cab1a05fcf8c824b9 P 0e32c19034a34c04b2e5f339a9143b69: Neither blocks nor log segments found. Creating new log.
I20260808 16:52:35.255479 20413 tablet_service.cc:1467] Processing CreateTablet for tablet 0c1801d6f33c463cab1a05fcf8c824b9 (DEFAULT_TABLE table=test-table [id=4f5597de33ce4d7ab5c5539abee900af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:52:35.255584 20413 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c1801d6f33c463cab1a05fcf8c824b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:52:35.256445 21030 tablet_bootstrap.cc:492] T 0c1801d6f33c463cab1a05fcf8c824b9 P 2771891c9a064fd6b6f4de95834948dc: Bootstrap starting.
I20260808 16:52:35.256609 20159 catalog_manager.cc:4841] TS 7962ba67a970456781409e433168be76 (127.8.24.2:37649): tablet 3aeac13b872e400ea727f0b888b48422 (table test-table [id=5b6bb7916bcf4c07bec5b3b399b272fc]) successfully deleted
I20260808 16:52:35.256681 21032 tablet_bootstrap.cc:492] T 0c1801d6f33c463cab1a05fcf8c824b9 P 7962ba67a970456781409e433168be76: No bootstrap required, opened a new log
I20260808 16:52:35.256724 21032 ts_tablet_manager.cc:1397] T 0c1801d6f33c463cab1a05fcf8c824b9 P 7962ba67a970456781409e433168be76: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 16:52:35.256839 21030 tablet_bootstrap.cc:654] T 0c1801d6f33c463cab1a05fcf8c824b9 P 2771891c9a064fd6b6f4de95834948dc: Neither blocks nor log segments found. Creating new log.
I20260808 16:52:35.256865 21032 raft_consensus.cc:348] T 0c1801d6f33c463cab1a05fcf8c824b9 P 7962ba67a970456781409e433168be76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2771891c9a064fd6b6f4de95834948dc" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 45233 } } peers { permanent_uuid: "7962ba67a970456781409e433168be76" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 37649 } } peers { permanent_uuid: "0e32c19034a34c04b2e5f339a9143b69" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 34921 } }
I20260808 16:52:35.256924 21032 raft_consensus.cc:374] T 0c1801d6f33c463cab1a05fcf8c824b9 P 7962ba67a970456781409e433168be76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:52:35.256965 21032 raft_consensus.cc:729] T 0c1801d6f33c463cab1a05fcf8c824b9 P 7962ba67a970456781409e433168be76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7962ba67a970456781409e433168be76, State: Initialized, Role: FOLLOWER
I20260808 16:52:35.257007 21032 consensus_queue.cc:260] T 0c1801d6f33c463cab1a05fcf8c824b9 P 7962ba67a970456781409e433168be76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2771891c9a064fd6b6f4de95834948dc" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 45233 } } peers { permanent_uuid: "7962ba67a970456781409e433168be76" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 37649 } } peers { permanent_uuid: "0e32c19034a34c04b2e5f339a9143b69" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 34921 } }
I20260808 16:52:35.257104 21032 ts_tablet_manager.cc:1428] T 0c1801d6f33c463cab1a05fcf8c824b9 P 7962ba67a970456781409e433168be76: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 16:52:35.257551 21030 tablet_bootstrap.cc:492] T 0c1801d6f33c463cab1a05fcf8c824b9 P 2771891c9a064fd6b6f4de95834948dc: No bootstrap required, opened a new log
I20260808 16:52:35.257565 21031 tablet_bootstrap.cc:492] T 0c1801d6f33c463cab1a05fcf8c824b9 P 0e32c19034a34c04b2e5f339a9143b69: No bootstrap required, opened a new log
I20260808 16:52:35.257601 21030 ts_tablet_manager.cc:1397] T 0c1801d6f33c463cab1a05fcf8c824b9 P 2771891c9a064fd6b6f4de95834948dc: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 16:52:35.257607 21031 ts_tablet_manager.cc:1397] T 0c1801d6f33c463cab1a05fcf8c824b9 P 0e32c19034a34c04b2e5f339a9143b69: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260808 16:52:35.257743 21030 raft_consensus.cc:348] T 0c1801d6f33c463cab1a05fcf8c824b9 P 2771891c9a064fd6b6f4de95834948dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2771891c9a064fd6b6f4de95834948dc" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 45233 } } peers { permanent_uuid: "7962ba67a970456781409e433168be76" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 37649 } } peers { permanent_uuid: "0e32c19034a34c04b2e5f339a9143b69" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 34921 } }
I20260808 16:52:35.257756 21031 raft_consensus.cc:348] T 0c1801d6f33c463cab1a05fcf8c824b9 P 0e32c19034a34c04b2e5f339a9143b69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2771891c9a064fd6b6f4de95834948dc" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 45233 } } peers { permanent_uuid: "7962ba67a970456781409e433168be76" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 37649 } } peers { permanent_uuid: "0e32c19034a34c04b2e5f339a9143b69" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 34921 } }
I20260808 16:52:35.257807 21030 raft_consensus.cc:374] T 0c1801d6f33c463cab1a05fcf8c824b9 P 2771891c9a064fd6b6f4de95834948dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:52:35.257824 21031 raft_consensus.cc:374] T 0c1801d6f33c463cab1a05fcf8c824b9 P 0e32c19034a34c04b2e5f339a9143b69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:52:35.257828 21030 raft_consensus.cc:729] T 0c1801d6f33c463cab1a05fcf8c824b9 P 2771891c9a064fd6b6f4de95834948dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2771891c9a064fd6b6f4de95834948dc, State: Initialized, Role: FOLLOWER
I20260808 16:52:35.257844 21031 raft_consensus.cc:729] T 0c1801d6f33c463cab1a05fcf8c824b9 P 0e32c19034a34c04b2e5f339a9143b69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e32c19034a34c04b2e5f339a9143b69, State: Initialized, Role: FOLLOWER
I20260808 16:52:35.257866 21030 consensus_queue.cc:260] T 0c1801d6f33c463cab1a05fcf8c824b9 P 2771891c9a064fd6b6f4de95834948dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2771891c9a064fd6b6f4de95834948dc" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 45233 } } peers { permanent_uuid: "7962ba67a970456781409e433168be76" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 37649 } } peers { permanent_uuid: "0e32c19034a34c04b2e5f339a9143b69" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 34921 } }
I20260808 16:52:35.257961 21030 ts_tablet_manager.cc:1428] T 0c1801d6f33c463cab1a05fcf8c824b9 P 2771891c9a064fd6b6f4de95834948dc: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:35.257884 21031 consensus_queue.cc:260] T 0c1801d6f33c463cab1a05fcf8c824b9 P 0e32c19034a34c04b2e5f339a9143b69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2771891c9a064fd6b6f4de95834948dc" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 45233 } } peers { permanent_uuid: "7962ba67a970456781409e433168be76" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 37649 } } peers { permanent_uuid: "0e32c19034a34c04b2e5f339a9143b69" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 34921 } }
I20260808 16:52:35.258468 21031 ts_tablet_manager.cc:1428] T 0c1801d6f33c463cab1a05fcf8c824b9 P 0e32c19034a34c04b2e5f339a9143b69: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 16:52:35.262486 21064 tablet_replica.cc:330] T 3aeac13b872e400ea727f0b888b48422 P 2771891c9a064fd6b6f4de95834948dc: stopping tablet replica
I20260808 16:52:35.262586 21064 raft_consensus.cc:2229] T 3aeac13b872e400ea727f0b888b48422 P 2771891c9a064fd6b6f4de95834948dc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 16:52:35.262697 21064 raft_consensus.cc:2258] T 3aeac13b872e400ea727f0b888b48422 P 2771891c9a064fd6b6f4de95834948dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:52:35.262842 21063 tablet_replica.cc:330] T 3aeac13b872e400ea727f0b888b48422 P 0e32c19034a34c04b2e5f339a9143b69: stopping tablet replica
I20260808 16:52:35.262919 21063 raft_consensus.cc:2229] T 3aeac13b872e400ea727f0b888b48422 P 0e32c19034a34c04b2e5f339a9143b69 [term 1 LEADER]: Raft consensus shutting down.
I20260808 16:52:35.263017 21063 raft_consensus.cc:2258] T 3aeac13b872e400ea727f0b888b48422 P 0e32c19034a34c04b2e5f339a9143b69 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:52:35.263355 21064 ts_tablet_manager.cc:1905] T 3aeac13b872e400ea727f0b888b48422 P 2771891c9a064fd6b6f4de95834948dc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:52:35.263422 21063 ts_tablet_manager.cc:1905] T 3aeac13b872e400ea727f0b888b48422 P 0e32c19034a34c04b2e5f339a9143b69: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:52:35.264796 21063 ts_tablet_manager.cc:1918] T 3aeac13b872e400ea727f0b888b48422 P 0e32c19034a34c04b2e5f339a9143b69: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 16:52:35.264868 21063 log.cc:1196] T 3aeac13b872e400ea727f0b888b48422 P 0e32c19034a34c04b2e5f339a9143b69: Deleting WAL directory at /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/minicluster-data/ts-2/wal/wals/3aeac13b872e400ea727f0b888b48422
I20260808 16:52:35.265038 21064 ts_tablet_manager.cc:1918] T 3aeac13b872e400ea727f0b888b48422 P 2771891c9a064fd6b6f4de95834948dc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 16:52:35.265106 21064 log.cc:1196] T 3aeac13b872e400ea727f0b888b48422 P 2771891c9a064fd6b6f4de95834948dc: Deleting WAL directory at /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/minicluster-data/ts-0/wal/wals/3aeac13b872e400ea727f0b888b48422
I20260808 16:52:35.265105 21063 ts_tablet_manager.cc:1939] T 3aeac13b872e400ea727f0b888b48422 P 0e32c19034a34c04b2e5f339a9143b69: Deleting consensus metadata
I20260808 16:52:35.265336 21064 ts_tablet_manager.cc:1939] T 3aeac13b872e400ea727f0b888b48422 P 2771891c9a064fd6b6f4de95834948dc: Deleting consensus metadata
I20260808 16:52:35.265712 20163 catalog_manager.cc:4841] TS 2771891c9a064fd6b6f4de95834948dc (127.8.24.1:45233): tablet 3aeac13b872e400ea727f0b888b48422 (table test-table [id=5b6bb7916bcf4c07bec5b3b399b272fc]) successfully deleted
I20260808 16:52:35.265791 20163 catalog_manager.cc:4841] TS 0e32c19034a34c04b2e5f339a9143b69 (127.8.24.3:34921): tablet 3aeac13b872e400ea727f0b888b48422 (table test-table [id=5b6bb7916bcf4c07bec5b3b399b272fc]) successfully deleted
I20260808 16:52:35.287326 21037 raft_consensus.cc:482] T 0c1801d6f33c463cab1a05fcf8c824b9 P 2771891c9a064fd6b6f4de95834948dc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 16:52:35.287395 21037 raft_consensus.cc:504] T 0c1801d6f33c463cab1a05fcf8c824b9 P 2771891c9a064fd6b6f4de95834948dc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2771891c9a064fd6b6f4de95834948dc" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 45233 } } peers { permanent_uuid: "7962ba67a970456781409e433168be76" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 37649 } } peers { permanent_uuid: "0e32c19034a34c04b2e5f339a9143b69" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 34921 } }
I20260808 16:52:35.287643 21037 leader_election.cc:288] T 0c1801d6f33c463cab1a05fcf8c824b9 P 2771891c9a064fd6b6f4de95834948dc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7962ba67a970456781409e433168be76 (127.8.24.2:37649), 0e32c19034a34c04b2e5f339a9143b69 (127.8.24.3:34921)
I20260808 16:52:35.290846 21025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:35.287844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:43235 (local address 127.8.24.3:34921)
0808 16:52:35.287897 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:35.287898 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:35.287956 (+ 58us) server_negotiation.cc:407] Connection header received
0808 16:52:35.288006 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:35.288007 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:35.288014 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:35.288068 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:52:35.288196 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:35.288882 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:35.290136 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:35.290540 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:35.290600 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:35.290609 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:35.290764 (+ 155us) server_negotiation.cc:299] Negotiation successful
0808 16:52:35.290794 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 16:52:35.290859 21072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:35.287755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.3:34921 (local address 127.8.24.1:43235)
0808 16:52:35.287890 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:35.287900 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:35.287958 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:35.288076 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:35.288076 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:35.288077 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:52:35.288139 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:35.288141 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:35.288957 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:35.288958 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:35.290023 (+ 1065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:35.290025 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:35.290610 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:35.290618 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:35.290698 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:35.290701 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 16:52:35.290750 (+ 49us) client_negotiation.cc:240] Negotiation successful
0808 16:52:35.290787 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":43,"threads_started":1}
I20260808 16:52:35.291127 21027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:35.287835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:60479 (local address 127.8.24.2:37649)
0808 16:52:35.287868 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:35.287870 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:35.288029 (+ 159us) server_negotiation.cc:407] Connection header received
0808 16:52:35.288080 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:35.288081 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:35.288090 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:35.288135 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:52:35.288305 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:35.288741 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:35.290115 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:35.290718 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:35.290783 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:35.290791 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:35.291033 (+ 242us) server_negotiation.cc:299] Negotiation successful
0808 16:52:35.291070 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 16:52:35.291095 20908 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c1801d6f33c463cab1a05fcf8c824b9" candidate_uuid: "2771891c9a064fd6b6f4de95834948dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e32c19034a34c04b2e5f339a9143b69" is_pre_election: true
I20260808 16:52:35.291196 20908 raft_consensus.cc:2454] T 0c1801d6f33c463cab1a05fcf8c824b9 P 0e32c19034a34c04b2e5f339a9143b69 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2771891c9a064fd6b6f4de95834948dc in term 0.
I20260808 16:52:35.291369 21073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:35.287755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.2:37649 (local address 127.8.24.1:60479)
0808 16:52:35.287974 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:35.287982 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:35.288026 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:35.288190 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:35.288190 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:35.288192 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:52:35.288242 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:35.288243 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:35.288809 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:35.288810 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:35.290020 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:35.290023 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:35.290876 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:35.290877 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:35.290952 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:35.290956 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 16:52:35.291244 (+ 288us) client_negotiation.cc:240] Negotiation successful
0808 16:52:35.291285 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"mutex_wait_us":2,"thread_start_us":42,"threads_started":1}
I20260808 16:52:35.291368 20337 leader_election.cc:302] T 0c1801d6f33c463cab1a05fcf8c824b9 P 2771891c9a064fd6b6f4de95834948dc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e32c19034a34c04b2e5f339a9143b69, 2771891c9a064fd6b6f4de95834948dc; no voters:
I20260808 16:52:35.291527 21037 raft_consensus.cc:2781] T 0c1801d6f33c463cab1a05fcf8c824b9 P 2771891c9a064fd6b6f4de95834948dc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 16:52:35.291556 21037 raft_consensus.cc:482] T 0c1801d6f33c463cab1a05fcf8c824b9 P 2771891c9a064fd6b6f4de95834948dc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 16:52:35.291577 21037 raft_consensus.cc:3037] T 0c1801d6f33c463cab1a05fcf8c824b9 P 2771891c9a064fd6b6f4de95834948dc [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:52:35.291589 20664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c1801d6f33c463cab1a05fcf8c824b9" candidate_uuid: "2771891c9a064fd6b6f4de95834948dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7962ba67a970456781409e433168be76" is_pre_election: true
I20260808 16:52:35.291657 20664 raft_consensus.cc:2454] T 0c1801d6f33c463cab1a05fcf8c824b9 P 7962ba67a970456781409e433168be76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2771891c9a064fd6b6f4de95834948dc in term 0.
I20260808 16:52:35.292158 21037 raft_consensus.cc:504] T 0c1801d6f33c463cab1a05fcf8c824b9 P 2771891c9a064fd6b6f4de95834948dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2771891c9a064fd6b6f4de95834948dc" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 45233 } } peers { permanent_uuid: "7962ba67a970456781409e433168be76" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 37649 } } peers { permanent_uuid: "0e32c19034a34c04b2e5f339a9143b69" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 34921 } }
I20260808 16:52:35.292275 21037 leader_election.cc:288] T 0c1801d6f33c463cab1a05fcf8c824b9 P 2771891c9a064fd6b6f4de95834948dc [CANDIDATE]: Term 1 election: Requested vote from peers 7962ba67a970456781409e433168be76 (127.8.24.2:37649), 0e32c19034a34c04b2e5f339a9143b69 (127.8.24.3:34921)
I20260808 16:52:35.292447 20664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c1801d6f33c463cab1a05fcf8c824b9" candidate_uuid: "2771891c9a064fd6b6f4de95834948dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7962ba67a970456781409e433168be76"
I20260808 16:52:35.292514 20664 raft_consensus.cc:3037] T 0c1801d6f33c463cab1a05fcf8c824b9 P 7962ba67a970456781409e433168be76 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:52:35.293277 20664 raft_consensus.cc:2454] T 0c1801d6f33c463cab1a05fcf8c824b9 P 7962ba67a970456781409e433168be76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2771891c9a064fd6b6f4de95834948dc in term 1.
I20260808 16:52:35.293391 20908 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c1801d6f33c463cab1a05fcf8c824b9" candidate_uuid: "2771891c9a064fd6b6f4de95834948dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e32c19034a34c04b2e5f339a9143b69"
I20260808 16:52:35.293442 20331 leader_election.cc:302] T 0c1801d6f33c463cab1a05fcf8c824b9 P 2771891c9a064fd6b6f4de95834948dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2771891c9a064fd6b6f4de95834948dc, 7962ba67a970456781409e433168be76; no voters:
I20260808 16:52:35.293458 20908 raft_consensus.cc:3037] T 0c1801d6f33c463cab1a05fcf8c824b9 P 0e32c19034a34c04b2e5f339a9143b69 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:52:35.293507 21037 raft_consensus.cc:2781] T 0c1801d6f33c463cab1a05fcf8c824b9 P 2771891c9a064fd6b6f4de95834948dc [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:52:35.293632 21037 raft_consensus.cc:686] T 0c1801d6f33c463cab1a05fcf8c824b9 P 2771891c9a064fd6b6f4de95834948dc [term 1 LEADER]: Becoming Leader. State: Replica: 2771891c9a064fd6b6f4de95834948dc, State: Running, Role: LEADER
I20260808 16:52:35.293697 21037 consensus_queue.cc:237] T 0c1801d6f33c463cab1a05fcf8c824b9 P 2771891c9a064fd6b6f4de95834948dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2771891c9a064fd6b6f4de95834948dc" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 45233 } } peers { permanent_uuid: "7962ba67a970456781409e433168be76" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 37649 } } peers { permanent_uuid: "0e32c19034a34c04b2e5f339a9143b69" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 34921 } }
I20260808 16:52:35.294122 20908 raft_consensus.cc:2454] T 0c1801d6f33c463cab1a05fcf8c824b9 P 0e32c19034a34c04b2e5f339a9143b69 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2771891c9a064fd6b6f4de95834948dc in term 1.
I20260808 16:52:35.294241 20179 catalog_manager.cc:5495] T 0c1801d6f33c463cab1a05fcf8c824b9 P 2771891c9a064fd6b6f4de95834948dc reported cstate change: term changed from 0 to 1, leader changed from <none> to 2771891c9a064fd6b6f4de95834948dc (127.8.24.1). New cstate: current_term: 1 leader_uuid: "2771891c9a064fd6b6f4de95834948dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2771891c9a064fd6b6f4de95834948dc" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 45233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7962ba67a970456781409e433168be76" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 37649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e32c19034a34c04b2e5f339a9143b69" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 34921 } health_report { overall_health: UNKNOWN } } }
Aug 08 16:52:35 dist-test-slave-6zbq krb5kdc[20112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207954, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.24.62@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM
I20260808 16:52:35.331156 21088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:35.324781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:39641 (local address 127.0.0.1:55418)
0808 16:52:35.324896 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:35.324907 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:35.324973 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:35.325120 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:35.325121 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:35.325396 (+ 275us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:35.325435 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:35.325437 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:35.326059 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:35.326060 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:35.326608 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:35.326610 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:35.326912 (+ 302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:35.326913 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:35.326958 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:35.327496 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:35.327501 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:35.328746 (+ 1245us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:52:35.330170 (+ 1424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:35.330177 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:35.330179 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:35.330394 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:35.330559 (+ 165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:35.330560 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:35.330560 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:35.330616 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:35.330813 (+ 197us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:52:35.330821 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:52:35.330864 (+ 43us) client_negotiation.cc:769] Sending connection context
0808 16:52:35.330923 (+ 59us) client_negotiation.cc:240] Negotiation successful
0808 16:52:35.331030 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":55,"threads_started":1}
I20260808 16:52:35.331164 21089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:35.324817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55418 (local address 127.8.24.62:39641)
0808 16:52:35.324903 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:35.324905 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:35.324964 (+ 59us) server_negotiation.cc:407] Connection header received
0808 16:52:35.325026 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:35.325027 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:35.325037 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:35.325093 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:35.325512 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:35.325992 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:35.326685 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:35.326859 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:35.326909 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:35.328822 (+ 1913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:35.328824 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:35.328824 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:35.328830 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:35.330099 (+ 1269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:35.330450 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:35.330451 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:35.330451 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:35.330506 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:35.330665 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:35.330666 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:35.330667 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:35.330760 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:35.330813 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:35.330951 (+ 138us) server_negotiation.cc:299] Negotiation successful
0808 16:52:35.331080 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":67,"server-negotiator.queue_time_us":73,"thread_start_us":41,"threads_started":1}
I20260808 16:52:35.332988 20179 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:55418:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 16:52:35.333161 20179 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 16:52:35.334565 20873 tablet_service.cc:1467] Processing CreateTablet for tablet ce4a8cd8a87f42db82bd77082624755f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b6999fa9a487434292397f91c104cc63]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 16:52:35.334580 20634 tablet_service.cc:1467] Processing CreateTablet for tablet ce4a8cd8a87f42db82bd77082624755f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b6999fa9a487434292397f91c104cc63]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 16:52:35.334571 20413 tablet_service.cc:1467] Processing CreateTablet for tablet ce4a8cd8a87f42db82bd77082624755f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b6999fa9a487434292397f91c104cc63]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 16:52:35.334709 20413 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce4a8cd8a87f42db82bd77082624755f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:52:35.334709 20634 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce4a8cd8a87f42db82bd77082624755f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:52:35.334712 20873 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce4a8cd8a87f42db82bd77082624755f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:52:35.336130 21031 tablet_bootstrap.cc:492] T ce4a8cd8a87f42db82bd77082624755f P 0e32c19034a34c04b2e5f339a9143b69: Bootstrap starting.
I20260808 16:52:35.336145 21032 tablet_bootstrap.cc:492] T ce4a8cd8a87f42db82bd77082624755f P 7962ba67a970456781409e433168be76: Bootstrap starting.
I20260808 16:52:35.336477 21032 tablet_bootstrap.cc:654] T ce4a8cd8a87f42db82bd77082624755f P 7962ba67a970456781409e433168be76: Neither blocks nor log segments found. Creating new log.
I20260808 16:52:35.336526 21031 tablet_bootstrap.cc:654] T ce4a8cd8a87f42db82bd77082624755f P 0e32c19034a34c04b2e5f339a9143b69: Neither blocks nor log segments found. Creating new log.
I20260808 16:52:35.336812 21030 tablet_bootstrap.cc:492] T ce4a8cd8a87f42db82bd77082624755f P 2771891c9a064fd6b6f4de95834948dc: Bootstrap starting.
I20260808 16:52:35.337080 21032 tablet_bootstrap.cc:492] T ce4a8cd8a87f42db82bd77082624755f P 7962ba67a970456781409e433168be76: No bootstrap required, opened a new log
I20260808 16:52:35.337131 21032 ts_tablet_manager.cc:1397] T ce4a8cd8a87f42db82bd77082624755f P 7962ba67a970456781409e433168be76: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 16:52:35.337193 21030 tablet_bootstrap.cc:654] T ce4a8cd8a87f42db82bd77082624755f P 2771891c9a064fd6b6f4de95834948dc: Neither blocks nor log segments found. Creating new log.
I20260808 16:52:35.337229 21032 raft_consensus.cc:348] T ce4a8cd8a87f42db82bd77082624755f P 7962ba67a970456781409e433168be76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2771891c9a064fd6b6f4de95834948dc" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 45233 } } peers { permanent_uuid: "7962ba67a970456781409e433168be76" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 37649 } } peers { permanent_uuid: "0e32c19034a34c04b2e5f339a9143b69" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 34921 } }
I20260808 16:52:35.337272 21032 raft_consensus.cc:374] T ce4a8cd8a87f42db82bd77082624755f P 7962ba67a970456781409e433168be76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:52:35.337282 21032 raft_consensus.cc:729] T ce4a8cd8a87f42db82bd77082624755f P 7962ba67a970456781409e433168be76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7962ba67a970456781409e433168be76, State: Initialized, Role: FOLLOWER
I20260808 16:52:35.337319 21032 consensus_queue.cc:260] T ce4a8cd8a87f42db82bd77082624755f P 7962ba67a970456781409e433168be76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2771891c9a064fd6b6f4de95834948dc" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 45233 } } peers { permanent_uuid: "7962ba67a970456781409e433168be76" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 37649 } } peers { permanent_uuid: "0e32c19034a34c04b2e5f339a9143b69" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 34921 } }
I20260808 16:52:35.337399 21032 ts_tablet_manager.cc:1428] T ce4a8cd8a87f42db82bd77082624755f P 7962ba67a970456781409e433168be76: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:35.337450 21040 tablet_replica.cc:439] T ce4a8cd8a87f42db82bd77082624755f P 7962ba67a970456781409e433168be76: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2771891c9a064fd6b6f4de95834948dc" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 45233 } } peers { permanent_uuid: "7962ba67a970456781409e433168be76" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 37649 } } peers { permanent_uuid: "0e32c19034a34c04b2e5f339a9143b69" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 34921 } } }
I20260808 16:52:35.337515 21040 tablet_replica.cc:442] T ce4a8cd8a87f42db82bd77082624755f P 7962ba67a970456781409e433168be76: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 16:52:35.337863 21030 tablet_bootstrap.cc:492] T ce4a8cd8a87f42db82bd77082624755f P 2771891c9a064fd6b6f4de95834948dc: No bootstrap required, opened a new log
I20260808 16:52:35.337919 21030 ts_tablet_manager.cc:1397] T ce4a8cd8a87f42db82bd77082624755f P 2771891c9a064fd6b6f4de95834948dc: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 16:52:35.338063 21030 raft_consensus.cc:348] T ce4a8cd8a87f42db82bd77082624755f P 2771891c9a064fd6b6f4de95834948dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2771891c9a064fd6b6f4de95834948dc" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 45233 } } peers { permanent_uuid: "7962ba67a970456781409e433168be76" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 37649 } } peers { permanent_uuid: "0e32c19034a34c04b2e5f339a9143b69" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 34921 } }
I20260808 16:52:35.338135 21030 raft_consensus.cc:374] T ce4a8cd8a87f42db82bd77082624755f P 2771891c9a064fd6b6f4de95834948dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:52:35.338152 21030 raft_consensus.cc:729] T ce4a8cd8a87f42db82bd77082624755f P 2771891c9a064fd6b6f4de95834948dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2771891c9a064fd6b6f4de95834948dc, State: Initialized, Role: FOLLOWER
I20260808 16:52:35.338196 21030 consensus_queue.cc:260] T ce4a8cd8a87f42db82bd77082624755f P 2771891c9a064fd6b6f4de95834948dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2771891c9a064fd6b6f4de95834948dc" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 45233 } } peers { permanent_uuid: "7962ba67a970456781409e433168be76" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 37649 } } peers { permanent_uuid: "0e32c19034a34c04b2e5f339a9143b69" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 34921 } }
I20260808 16:52:35.338281 21030 ts_tablet_manager.cc:1428] T ce4a8cd8a87f42db82bd77082624755f P 2771891c9a064fd6b6f4de95834948dc: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 16:52:35.338400 21031 tablet_bootstrap.cc:492] T ce4a8cd8a87f42db82bd77082624755f P 0e32c19034a34c04b2e5f339a9143b69: No bootstrap required, opened a new log
I20260808 16:52:35.338398 21037 tablet_replica.cc:439] T ce4a8cd8a87f42db82bd77082624755f P 2771891c9a064fd6b6f4de95834948dc: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2771891c9a064fd6b6f4de95834948dc" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 45233 } } peers { permanent_uuid: "7962ba67a970456781409e433168be76" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 37649 } } peers { permanent_uuid: "0e32c19034a34c04b2e5f339a9143b69" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 34921 } } }
I20260808 16:52:35.338443 21031 ts_tablet_manager.cc:1397] T ce4a8cd8a87f42db82bd77082624755f P 0e32c19034a34c04b2e5f339a9143b69: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 16:52:35.338461 21037 tablet_replica.cc:442] T ce4a8cd8a87f42db82bd77082624755f P 2771891c9a064fd6b6f4de95834948dc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 16:52:35.338696 21031 raft_consensus.cc:348] T ce4a8cd8a87f42db82bd77082624755f P 0e32c19034a34c04b2e5f339a9143b69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2771891c9a064fd6b6f4de95834948dc" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 45233 } } peers { permanent_uuid: "7962ba67a970456781409e433168be76" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 37649 } } peers { permanent_uuid: "0e32c19034a34c04b2e5f339a9143b69" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 34921 } }
I20260808 16:52:35.338766 21031 raft_consensus.cc:374] T ce4a8cd8a87f42db82bd77082624755f P 0e32c19034a34c04b2e5f339a9143b69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:52:35.338785 21031 raft_consensus.cc:729] T ce4a8cd8a87f42db82bd77082624755f P 0e32c19034a34c04b2e5f339a9143b69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e32c19034a34c04b2e5f339a9143b69, State: Initialized, Role: FOLLOWER
I20260808 16:52:35.338829 21031 consensus_queue.cc:260] T ce4a8cd8a87f42db82bd77082624755f P 0e32c19034a34c04b2e5f339a9143b69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2771891c9a064fd6b6f4de95834948dc" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 45233 } } peers { permanent_uuid: "7962ba67a970456781409e433168be76" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 37649 } } peers { permanent_uuid: "0e32c19034a34c04b2e5f339a9143b69" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 34921 } }
I20260808 16:52:35.338908 21031 ts_tablet_manager.cc:1428] T ce4a8cd8a87f42db82bd77082624755f P 0e32c19034a34c04b2e5f339a9143b69: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 16:52:35.339002 21054 tablet_replica.cc:439] T ce4a8cd8a87f42db82bd77082624755f P 0e32c19034a34c04b2e5f339a9143b69: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2771891c9a064fd6b6f4de95834948dc" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 45233 } } peers { permanent_uuid: "7962ba67a970456781409e433168be76" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 37649 } } peers { permanent_uuid: "0e32c19034a34c04b2e5f339a9143b69" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 34921 } } }
I20260808 16:52:35.339057 21054 tablet_replica.cc:442] T ce4a8cd8a87f42db82bd77082624755f P 0e32c19034a34c04b2e5f339a9143b69: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 16:52:35.464782 21037 raft_consensus.cc:482] T ce4a8cd8a87f42db82bd77082624755f P 2771891c9a064fd6b6f4de95834948dc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 16:52:35.464865 21037 raft_consensus.cc:504] T ce4a8cd8a87f42db82bd77082624755f P 2771891c9a064fd6b6f4de95834948dc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2771891c9a064fd6b6f4de95834948dc" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 45233 } } peers { permanent_uuid: "7962ba67a970456781409e433168be76" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 37649 } } peers { permanent_uuid: "0e32c19034a34c04b2e5f339a9143b69" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 34921 } }
I20260808 16:52:35.464996 21037 leader_election.cc:288] T ce4a8cd8a87f42db82bd77082624755f P 2771891c9a064fd6b6f4de95834948dc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7962ba67a970456781409e433168be76 (127.8.24.2:37649), 0e32c19034a34c04b2e5f339a9143b69 (127.8.24.3:34921)
I20260808 16:52:35.465195 20664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce4a8cd8a87f42db82bd77082624755f" candidate_uuid: "2771891c9a064fd6b6f4de95834948dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7962ba67a970456781409e433168be76" is_pre_election: true
I20260808 16:52:35.465199 20908 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce4a8cd8a87f42db82bd77082624755f" candidate_uuid: "2771891c9a064fd6b6f4de95834948dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e32c19034a34c04b2e5f339a9143b69" is_pre_election: true
I20260808 16:52:35.465301 20664 raft_consensus.cc:2454] T ce4a8cd8a87f42db82bd77082624755f P 7962ba67a970456781409e433168be76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2771891c9a064fd6b6f4de95834948dc in term 0.
I20260808 16:52:35.465301 20908 raft_consensus.cc:2454] T ce4a8cd8a87f42db82bd77082624755f P 0e32c19034a34c04b2e5f339a9143b69 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2771891c9a064fd6b6f4de95834948dc in term 0.
I20260808 16:52:35.465442 20337 leader_election.cc:302] T ce4a8cd8a87f42db82bd77082624755f P 2771891c9a064fd6b6f4de95834948dc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e32c19034a34c04b2e5f339a9143b69, 2771891c9a064fd6b6f4de95834948dc; no voters:
I20260808 16:52:35.465533 21037 raft_consensus.cc:2781] T ce4a8cd8a87f42db82bd77082624755f P 2771891c9a064fd6b6f4de95834948dc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 16:52:35.465572 21037 raft_consensus.cc:482] T ce4a8cd8a87f42db82bd77082624755f P 2771891c9a064fd6b6f4de95834948dc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 16:52:35.465595 21037 raft_consensus.cc:3037] T ce4a8cd8a87f42db82bd77082624755f P 2771891c9a064fd6b6f4de95834948dc [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:52:35.466169 21037 raft_consensus.cc:504] T ce4a8cd8a87f42db82bd77082624755f P 2771891c9a064fd6b6f4de95834948dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2771891c9a064fd6b6f4de95834948dc" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 45233 } } peers { permanent_uuid: "7962ba67a970456781409e433168be76" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 37649 } } peers { permanent_uuid: "0e32c19034a34c04b2e5f339a9143b69" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 34921 } }
I20260808 16:52:35.466276 21037 leader_election.cc:288] T ce4a8cd8a87f42db82bd77082624755f P 2771891c9a064fd6b6f4de95834948dc [CANDIDATE]: Term 1 election: Requested vote from peers 7962ba67a970456781409e433168be76 (127.8.24.2:37649), 0e32c19034a34c04b2e5f339a9143b69 (127.8.24.3:34921)
I20260808 16:52:35.466439 20908 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce4a8cd8a87f42db82bd77082624755f" candidate_uuid: "2771891c9a064fd6b6f4de95834948dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e32c19034a34c04b2e5f339a9143b69"
I20260808 16:52:35.466456 20664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce4a8cd8a87f42db82bd77082624755f" candidate_uuid: "2771891c9a064fd6b6f4de95834948dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7962ba67a970456781409e433168be76"
I20260808 16:52:35.466521 20908 raft_consensus.cc:3037] T ce4a8cd8a87f42db82bd77082624755f P 0e32c19034a34c04b2e5f339a9143b69 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:52:35.466521 20664 raft_consensus.cc:3037] T ce4a8cd8a87f42db82bd77082624755f P 7962ba67a970456781409e433168be76 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:52:35.467231 20664 raft_consensus.cc:2454] T ce4a8cd8a87f42db82bd77082624755f P 7962ba67a970456781409e433168be76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2771891c9a064fd6b6f4de95834948dc in term 1.
I20260808 16:52:35.467231 20908 raft_consensus.cc:2454] T ce4a8cd8a87f42db82bd77082624755f P 0e32c19034a34c04b2e5f339a9143b69 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2771891c9a064fd6b6f4de95834948dc in term 1.
I20260808 16:52:35.467396 20331 leader_election.cc:302] T ce4a8cd8a87f42db82bd77082624755f P 2771891c9a064fd6b6f4de95834948dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2771891c9a064fd6b6f4de95834948dc, 7962ba67a970456781409e433168be76; no voters:
I20260808 16:52:35.467518 21037 raft_consensus.cc:2781] T ce4a8cd8a87f42db82bd77082624755f P 2771891c9a064fd6b6f4de95834948dc [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:52:35.467576 21037 raft_consensus.cc:686] T ce4a8cd8a87f42db82bd77082624755f P 2771891c9a064fd6b6f4de95834948dc [term 1 LEADER]: Becoming Leader. State: Replica: 2771891c9a064fd6b6f4de95834948dc, State: Running, Role: LEADER
I20260808 16:52:35.467631 21037 consensus_queue.cc:237] T ce4a8cd8a87f42db82bd77082624755f P 2771891c9a064fd6b6f4de95834948dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2771891c9a064fd6b6f4de95834948dc" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 45233 } } peers { permanent_uuid: "7962ba67a970456781409e433168be76" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 37649 } } peers { permanent_uuid: "0e32c19034a34c04b2e5f339a9143b69" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 34921 } }
I20260808 16:52:35.467932 21076 tablet_replica.cc:439] T ce4a8cd8a87f42db82bd77082624755f P 2771891c9a064fd6b6f4de95834948dc: TxnStatusTablet state changed. Reason: New leader 2771891c9a064fd6b6f4de95834948dc. Latest consensus state: current_term: 1 leader_uuid: "2771891c9a064fd6b6f4de95834948dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2771891c9a064fd6b6f4de95834948dc" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 45233 } } peers { permanent_uuid: "7962ba67a970456781409e433168be76" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 37649 } } peers { permanent_uuid: "0e32c19034a34c04b2e5f339a9143b69" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 34921 } } }
I20260808 16:52:35.468022 21076 tablet_replica.cc:442] T ce4a8cd8a87f42db82bd77082624755f P 2771891c9a064fd6b6f4de95834948dc: This TxnStatusTablet replica's current role is: LEADER
I20260808 16:52:35.468210 21097 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 16:52:35.468214 20179 catalog_manager.cc:5495] T ce4a8cd8a87f42db82bd77082624755f P 2771891c9a064fd6b6f4de95834948dc reported cstate change: term changed from 0 to 1, leader changed from <none> to 2771891c9a064fd6b6f4de95834948dc (127.8.24.1). New cstate: current_term: 1 leader_uuid: "2771891c9a064fd6b6f4de95834948dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2771891c9a064fd6b6f4de95834948dc" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 45233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7962ba67a970456781409e433168be76" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 37649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e32c19034a34c04b2e5f339a9143b69" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 34921 } health_report { overall_health: UNKNOWN } } }
I20260808 16:52:35.468326 21097 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 16:52:35.482746 21029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:35.480193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49422 (local address 127.8.24.1:45233)
0808 16:52:35.480232 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:35.480234 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:35.480280 (+ 46us) server_negotiation.cc:407] Connection header received
0808 16:52:35.480372 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:35.480373 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:35.480387 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:35.480444 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 16:52:35.480568 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:35.481020 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:35.482097 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:35.482265 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:35.482305 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:35.482492 (+ 187us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 16:52:35.482575 (+ 83us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 16:52:35.482620 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:35.482687 (+ 67us) server_negotiation.cc:299] Negotiation successful
0808 16:52:35.482704 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 16:52:35.482813 21088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:35.480158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:45233 (local address 127.0.0.1:49422)
0808 16:52:35.480238 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:35.480248 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:35.480301 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:35.480461 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:35.480462 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:35.480463 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 16:52:35.480513 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:35.480516 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:35.481076 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:35.481077 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:35.482027 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:35.482029 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:35.482318 (+ 289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:35.482319 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:35.482394 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:35.482411 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 16:52:35.482626 (+ 215us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 16:52:35.482632 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 16:52:35.482684 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 16:52:35.482736 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260808 16:52:35.483841 20664 raft_consensus.cc:1261] T ce4a8cd8a87f42db82bd77082624755f P 7962ba67a970456781409e433168be76 [term 1 FOLLOWER]: Refusing update from remote peer 2771891c9a064fd6b6f4de95834948dc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:52:35.483911 20908 raft_consensus.cc:1261] T ce4a8cd8a87f42db82bd77082624755f P 0e32c19034a34c04b2e5f339a9143b69 [term 1 FOLLOWER]: Refusing update from remote peer 2771891c9a064fd6b6f4de95834948dc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:52:35.484062 21076 consensus_queue.cc:1035] T ce4a8cd8a87f42db82bd77082624755f P 2771891c9a064fd6b6f4de95834948dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "7962ba67a970456781409e433168be76" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 37649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:52:35.484164 21037 consensus_queue.cc:1035] T ce4a8cd8a87f42db82bd77082624755f P 2771891c9a064fd6b6f4de95834948dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e32c19034a34c04b2e5f339a9143b69" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 34921 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:52:35.484714 21040 tablet_replica.cc:439] T ce4a8cd8a87f42db82bd77082624755f P 7962ba67a970456781409e433168be76: TxnStatusTablet state changed. Reason: New leader 2771891c9a064fd6b6f4de95834948dc. Latest consensus state: current_term: 1 leader_uuid: "2771891c9a064fd6b6f4de95834948dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2771891c9a064fd6b6f4de95834948dc" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 45233 } } peers { permanent_uuid: "7962ba67a970456781409e433168be76" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 37649 } } peers { permanent_uuid: "0e32c19034a34c04b2e5f339a9143b69" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 34921 } } }
I20260808 16:52:35.484797 21040 tablet_replica.cc:442] T ce4a8cd8a87f42db82bd77082624755f P 7962ba67a970456781409e433168be76: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 16:52:35.484778 21054 tablet_replica.cc:439] T ce4a8cd8a87f42db82bd77082624755f P 0e32c19034a34c04b2e5f339a9143b69: TxnStatusTablet state changed. Reason: New leader 2771891c9a064fd6b6f4de95834948dc. Latest consensus state: current_term: 1 leader_uuid: "2771891c9a064fd6b6f4de95834948dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2771891c9a064fd6b6f4de95834948dc" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 45233 } } peers { permanent_uuid: "7962ba67a970456781409e433168be76" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 37649 } } peers { permanent_uuid: "0e32c19034a34c04b2e5f339a9143b69" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 34921 } } }
I20260808 16:52:35.484855 21054 tablet_replica.cc:442] T ce4a8cd8a87f42db82bd77082624755f P 0e32c19034a34c04b2e5f339a9143b69: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 16:52:35.485433 21037 tablet_replica.cc:439] T ce4a8cd8a87f42db82bd77082624755f P 2771891c9a064fd6b6f4de95834948dc: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2771891c9a064fd6b6f4de95834948dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2771891c9a064fd6b6f4de95834948dc" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 45233 } } peers { permanent_uuid: "7962ba67a970456781409e433168be76" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 37649 } } peers { permanent_uuid: "0e32c19034a34c04b2e5f339a9143b69" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 34921 } } }
I20260808 16:52:35.485512 21037 tablet_replica.cc:442] T ce4a8cd8a87f42db82bd77082624755f P 2771891c9a064fd6b6f4de95834948dc: This TxnStatusTablet replica's current role is: LEADER
I20260808 16:52:35.485697 21037 tablet_replica.cc:439] T ce4a8cd8a87f42db82bd77082624755f P 2771891c9a064fd6b6f4de95834948dc: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2771891c9a064fd6b6f4de95834948dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2771891c9a064fd6b6f4de95834948dc" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 45233 } } peers { permanent_uuid: "7962ba67a970456781409e433168be76" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 37649 } } peers { permanent_uuid: "0e32c19034a34c04b2e5f339a9143b69" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 34921 } } }
I20260808 16:52:35.485770 21037 tablet_replica.cc:442] T ce4a8cd8a87f42db82bd77082624755f P 2771891c9a064fd6b6f4de95834948dc: This TxnStatusTablet replica's current role is: LEADER
I20260808 16:52:35.485765 21054 tablet_replica.cc:439] T ce4a8cd8a87f42db82bd77082624755f P 0e32c19034a34c04b2e5f339a9143b69: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2771891c9a064fd6b6f4de95834948dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2771891c9a064fd6b6f4de95834948dc" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 45233 } } peers { permanent_uuid: "7962ba67a970456781409e433168be76" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 37649 } } peers { permanent_uuid: "0e32c19034a34c04b2e5f339a9143b69" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 34921 } } }
I20260808 16:52:35.485834 21054 tablet_replica.cc:442] T ce4a8cd8a87f42db82bd77082624755f P 0e32c19034a34c04b2e5f339a9143b69: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 16:52:35.485948 21048 mvcc.cc:205] Tried to move back new op lower bound from 7316307785660063744 to 7316307785595826176. Current Snapshot: MvccSnapshot[applied={T|T < 7316307785660063744}]
I20260808 16:52:35.486112 21040 tablet_replica.cc:439] T ce4a8cd8a87f42db82bd77082624755f P 7962ba67a970456781409e433168be76: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2771891c9a064fd6b6f4de95834948dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2771891c9a064fd6b6f4de95834948dc" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 45233 } } peers { permanent_uuid: "7962ba67a970456781409e433168be76" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 37649 } } peers { permanent_uuid: "0e32c19034a34c04b2e5f339a9143b69" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 34921 } } }
I20260808 16:52:35.486176 21040 tablet_replica.cc:442] T ce4a8cd8a87f42db82bd77082624755f P 7962ba67a970456781409e433168be76: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 16:52:35.488978 21029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:35.487020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49424 (local address 127.8.24.1:45233)
0808 16:52:35.487049 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:35.487051 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:35.487058 (+ 7us) server_negotiation.cc:407] Connection header received
0808 16:52:35.487126 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:35.487127 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:35.487136 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:35.487174 (+ 38us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 16:52:35.487321 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:35.487752 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:35.488433 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:35.488616 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:35.488665 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:35.488787 (+ 122us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 16:52:35.488822 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 16:52:35.488861 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:35.488910 (+ 49us) server_negotiation.cc:299] Negotiation successful
0808 16:52:35.488925 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260808 16:52:35.493296 21029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:35.491159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:48055 (local address 127.8.24.1:45233)
0808 16:52:35.491196 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:35.491198 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:35.491336 (+ 138us) server_negotiation.cc:407] Connection header received
0808 16:52:35.491404 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:35.491405 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:35.491413 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:35.491452 (+ 39us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 16:52:35.491578 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:35.491983 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:35.492689 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:35.492858 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:35.492908 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:35.493067 (+ 159us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 16:52:35.493096 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 16:52:35.493139 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:35.493185 (+ 46us) server_negotiation.cc:299] Negotiation successful
0808 16:52:35.493204 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":20}
I20260808 16:52:35.493466 21106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:35.491127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:45233 (local address 127.8.24.1:48055)
0808 16:52:35.491273 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:35.491282 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:35.491336 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:35.491466 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:35.491467 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:35.491468 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 16:52:35.491528 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:35.491529 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:35.492043 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:35.492044 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:35.492621 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:35.492623 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:35.492930 (+ 307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:35.492931 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:35.492993 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:35.493006 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 16:52:35.493144 (+ 138us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 16:52:35.493145 (+ 1us) client_negotiation.cc:769] Sending connection context
0808 16:52:35.493185 (+ 40us) client_negotiation.cc:240] Negotiation successful
0808 16:52:35.493221 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":80,"threads_started":1}
I20260808 16:52:35.495245 20908 raft_consensus.cc:1261] T 0c1801d6f33c463cab1a05fcf8c824b9 P 0e32c19034a34c04b2e5f339a9143b69 [term 1 FOLLOWER]: Refusing update from remote peer 2771891c9a064fd6b6f4de95834948dc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:52:35.495340 20664 raft_consensus.cc:1261] T 0c1801d6f33c463cab1a05fcf8c824b9 P 7962ba67a970456781409e433168be76 [term 1 FOLLOWER]: Refusing update from remote peer 2771891c9a064fd6b6f4de95834948dc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:52:35.495476 21108 consensus_queue.cc:1035] T 0c1801d6f33c463cab1a05fcf8c824b9 P 2771891c9a064fd6b6f4de95834948dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e32c19034a34c04b2e5f339a9143b69" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 34921 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:52:35.495561 21076 consensus_queue.cc:1035] T 0c1801d6f33c463cab1a05fcf8c824b9 P 2771891c9a064fd6b6f4de95834948dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "7962ba67a970456781409e433168be76" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 37649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:52:35.506214 20181 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55404:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 16:52:35.506289 20181 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55404:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 16:52:35.506842 20181 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e63efa3dec1242f980a4d3123bd3f739: Sending DeleteTablet for 3 replicas of tablet 0c1801d6f33c463cab1a05fcf8c824b9
I20260808 16:52:35.507076 20873 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c1801d6f33c463cab1a05fcf8c824b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:52:35 UTC) from {username='oryx', principal='oryx/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:41986
I20260808 16:52:35.507091 20410 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c1801d6f33c463cab1a05fcf8c824b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:52:35 UTC) from {username='oryx', principal='oryx/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:49408
I20260808 16:52:35.507177 21064 tablet_replica.cc:330] T 0c1801d6f33c463cab1a05fcf8c824b9 P 2771891c9a064fd6b6f4de95834948dc: stopping tablet replica
I20260808 16:52:35.507194 21063 tablet_replica.cc:330] T 0c1801d6f33c463cab1a05fcf8c824b9 P 0e32c19034a34c04b2e5f339a9143b69: stopping tablet replica
I20260808 16:52:35.507222 21064 raft_consensus.cc:2229] T 0c1801d6f33c463cab1a05fcf8c824b9 P 2771891c9a064fd6b6f4de95834948dc [term 1 LEADER]: Raft consensus shutting down.
I20260808 16:52:35.507246 21063 raft_consensus.cc:2229] T 0c1801d6f33c463cab1a05fcf8c824b9 P 0e32c19034a34c04b2e5f339a9143b69 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 16:52:35.507282 21063 raft_consensus.cc:2258] T 0c1801d6f33c463cab1a05fcf8c824b9 P 0e32c19034a34c04b2e5f339a9143b69 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:52:35.507297 21064 raft_consensus.cc:2258] T 0c1801d6f33c463cab1a05fcf8c824b9 P 2771891c9a064fd6b6f4de95834948dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:52:35.507601 21063 ts_tablet_manager.cc:1905] T 0c1801d6f33c463cab1a05fcf8c824b9 P 0e32c19034a34c04b2e5f339a9143b69: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:52:35.507634 21064 ts_tablet_manager.cc:1905] T 0c1801d6f33c463cab1a05fcf8c824b9 P 2771891c9a064fd6b6f4de95834948dc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:52:35.509053 21063 ts_tablet_manager.cc:1918] T 0c1801d6f33c463cab1a05fcf8c824b9 P 0e32c19034a34c04b2e5f339a9143b69: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 16:52:35.509053 21064 ts_tablet_manager.cc:1918] T 0c1801d6f33c463cab1a05fcf8c824b9 P 2771891c9a064fd6b6f4de95834948dc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 16:52:35.509114 21064 log.cc:1196] T 0c1801d6f33c463cab1a05fcf8c824b9 P 2771891c9a064fd6b6f4de95834948dc: Deleting WAL directory at /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/minicluster-data/ts-0/wal/wals/0c1801d6f33c463cab1a05fcf8c824b9
I20260808 16:52:35.509112 21063 log.cc:1196] T 0c1801d6f33c463cab1a05fcf8c824b9 P 0e32c19034a34c04b2e5f339a9143b69: Deleting WAL directory at /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/minicluster-data/ts-2/wal/wals/0c1801d6f33c463cab1a05fcf8c824b9
I20260808 16:52:35.509307 21063 ts_tablet_manager.cc:1939] T 0c1801d6f33c463cab1a05fcf8c824b9 P 0e32c19034a34c04b2e5f339a9143b69: Deleting consensus metadata
I20260808 16:52:35.509317 21064 ts_tablet_manager.cc:1939] T 0c1801d6f33c463cab1a05fcf8c824b9 P 2771891c9a064fd6b6f4de95834948dc: Deleting consensus metadata
I20260808 16:52:35.509624 20163 catalog_manager.cc:4841] TS 0e32c19034a34c04b2e5f339a9143b69 (127.8.24.3:34921): tablet 0c1801d6f33c463cab1a05fcf8c824b9 (table test-table [id=4f5597de33ce4d7ab5c5539abee900af]) successfully deleted
I20260808 16:52:35.509697 20163 catalog_manager.cc:4841] TS 2771891c9a064fd6b6f4de95834948dc (127.8.24.1:45233): tablet 0c1801d6f33c463cab1a05fcf8c824b9 (table test-table [id=4f5597de33ce4d7ab5c5539abee900af]) successfully deleted
I20260808 16:52:35.512378 20634 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c1801d6f33c463cab1a05fcf8c824b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:52:35 UTC) from {username='oryx', principal='oryx/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:48898
I20260808 16:52:35.512477 21062 tablet_replica.cc:330] T 0c1801d6f33c463cab1a05fcf8c824b9 P 7962ba67a970456781409e433168be76: stopping tablet replica
I20260808 16:52:35.512534 21062 raft_consensus.cc:2229] T 0c1801d6f33c463cab1a05fcf8c824b9 P 7962ba67a970456781409e433168be76 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 16:52:35.512567 21062 raft_consensus.cc:2258] T 0c1801d6f33c463cab1a05fcf8c824b9 P 7962ba67a970456781409e433168be76 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:52:35.512847 21062 ts_tablet_manager.cc:1905] T 0c1801d6f33c463cab1a05fcf8c824b9 P 7962ba67a970456781409e433168be76: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:52:35.514292 21062 ts_tablet_manager.cc:1918] T 0c1801d6f33c463cab1a05fcf8c824b9 P 7962ba67a970456781409e433168be76: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 16:52:35.514362 21062 log.cc:1196] T 0c1801d6f33c463cab1a05fcf8c824b9 P 7962ba67a970456781409e433168be76: Deleting WAL directory at /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/minicluster-data/ts-1/wal/wals/0c1801d6f33c463cab1a05fcf8c824b9
I20260808 16:52:35.514556 21062 ts_tablet_manager.cc:1939] T 0c1801d6f33c463cab1a05fcf8c824b9 P 7962ba67a970456781409e433168be76: Deleting consensus metadata
I20260808 16:52:35.514853 20159 catalog_manager.cc:4841] TS 7962ba67a970456781409e433168be76 (127.8.24.2:37649): tablet 0c1801d6f33c463cab1a05fcf8c824b9 (table test-table [id=4f5597de33ce4d7ab5c5539abee900af]) successfully deleted
Aug 08 16:52:35 dist-test-slave-6zbq krb5kdc[20112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207955, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 16:52:35 dist-test-slave-6zbq krb5kdc[20112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:52:35 dist-test-slave-6zbq krb5kdc[20112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
W20260808 16:52:35.531440 21089 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:52:35.525418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55428 (local address 127.8.24.62:39641)
0808 16:52:35.525476 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:35.525478 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:35.525582 (+ 104us) server_negotiation.cc:407] Connection header received
0808 16:52:35.525689 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:35.525690 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:35.525701 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:35.525780 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:35.526514 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:35.527354 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:35.528400 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:35.528680 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:35.528724 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:35.531382 (+ 2658us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55428: BlockingRecv error: recv got EOF from 127.0.0.1:55428 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Aug 08 16:52:35 dist-test-slave-6zbq krb5kdc[20112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207955, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM
I20260808 16:52:35.538399 21089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:35.532218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55432 (local address 127.8.24.62:39641)
0808 16:52:35.532264 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:35.532266 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:35.532304 (+ 38us) server_negotiation.cc:407] Connection header received
0808 16:52:35.532360 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:35.532361 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:35.532369 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:35.532408 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:35.532817 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:35.533248 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:35.533867 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:35.534052 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:35.534102 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:35.536254 (+ 2152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:35.536255 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:35.536255 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:35.536261 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:35.537428 (+ 1167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:35.537777 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:35.537778 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:35.537779 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:35.537820 (+ 41us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:35.537981 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:35.537981 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:35.537981 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:35.538064 (+ 83us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:35.538128 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:35.538249 (+ 121us) server_negotiation.cc:299] Negotiation successful
0808 16:52:35.538311 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 16:52:35.539681 20181 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55432:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 16:52:35.539829 20181 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 16:52:35.541019 20634 tablet_service.cc:1467] Processing CreateTablet for tablet 9ee546be89e54179954fe39c04018cc8 (DEFAULT_TABLE table=test-table [id=38b1314b8f404d8ca14161362f1b2358]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:52:35.541082 20410 tablet_service.cc:1467] Processing CreateTablet for tablet 9ee546be89e54179954fe39c04018cc8 (DEFAULT_TABLE table=test-table [id=38b1314b8f404d8ca14161362f1b2358]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:52:35.541150 20634 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ee546be89e54179954fe39c04018cc8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:52:35.541186 20410 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ee546be89e54179954fe39c04018cc8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:52:35.542513 21030 tablet_bootstrap.cc:492] T 9ee546be89e54179954fe39c04018cc8 P 2771891c9a064fd6b6f4de95834948dc: Bootstrap starting.
I20260808 16:52:35.542753 21032 tablet_bootstrap.cc:492] T 9ee546be89e54179954fe39c04018cc8 P 7962ba67a970456781409e433168be76: Bootstrap starting.
I20260808 16:52:35.542879 21030 tablet_bootstrap.cc:654] T 9ee546be89e54179954fe39c04018cc8 P 2771891c9a064fd6b6f4de95834948dc: Neither blocks nor log segments found. Creating new log.
I20260808 16:52:35.543107 21032 tablet_bootstrap.cc:654] T 9ee546be89e54179954fe39c04018cc8 P 7962ba67a970456781409e433168be76: Neither blocks nor log segments found. Creating new log.
I20260808 16:52:35.543270 20873 tablet_service.cc:1467] Processing CreateTablet for tablet 9ee546be89e54179954fe39c04018cc8 (DEFAULT_TABLE table=test-table [id=38b1314b8f404d8ca14161362f1b2358]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:52:35.543386 20873 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ee546be89e54179954fe39c04018cc8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:52:35.543608 21030 tablet_bootstrap.cc:492] T 9ee546be89e54179954fe39c04018cc8 P 2771891c9a064fd6b6f4de95834948dc: No bootstrap required, opened a new log
I20260808 16:52:35.543660 21030 ts_tablet_manager.cc:1397] T 9ee546be89e54179954fe39c04018cc8 P 2771891c9a064fd6b6f4de95834948dc: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 16:52:35.543812 21030 raft_consensus.cc:348] T 9ee546be89e54179954fe39c04018cc8 P 2771891c9a064fd6b6f4de95834948dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2771891c9a064fd6b6f4de95834948dc" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 45233 } } peers { permanent_uuid: "7962ba67a970456781409e433168be76" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 37649 } } peers { permanent_uuid: "0e32c19034a34c04b2e5f339a9143b69" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 34921 } }
I20260808 16:52:35.543879 21030 raft_consensus.cc:374] T 9ee546be89e54179954fe39c04018cc8 P 2771891c9a064fd6b6f4de95834948dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:52:35.543898 21030 raft_consensus.cc:729] T 9ee546be89e54179954fe39c04018cc8 P 2771891c9a064fd6b6f4de95834948dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2771891c9a064fd6b6f4de95834948dc, State: Initialized, Role: FOLLOWER
I20260808 16:52:35.543923 21032 tablet_bootstrap.cc:492] T 9ee546be89e54179954fe39c04018cc8 P 7962ba67a970456781409e433168be76: No bootstrap required, opened a new log
I20260808 16:52:35.543936 21030 consensus_queue.cc:260] T 9ee546be89e54179954fe39c04018cc8 P 2771891c9a064fd6b6f4de95834948dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2771891c9a064fd6b6f4de95834948dc" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 45233 } } peers { permanent_uuid: "7962ba67a970456781409e433168be76" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 37649 } } peers { permanent_uuid: "0e32c19034a34c04b2e5f339a9143b69" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 34921 } }
I20260808 16:52:35.543959 21032 ts_tablet_manager.cc:1397] T 9ee546be89e54179954fe39c04018cc8 P 7962ba67a970456781409e433168be76: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 16:52:35.544037 21030 ts_tablet_manager.cc:1428] T 9ee546be89e54179954fe39c04018cc8 P 2771891c9a064fd6b6f4de95834948dc: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 16:52:35.544091 21032 raft_consensus.cc:348] T 9ee546be89e54179954fe39c04018cc8 P 7962ba67a970456781409e433168be76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2771891c9a064fd6b6f4de95834948dc" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 45233 } } peers { permanent_uuid: "7962ba67a970456781409e433168be76" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 37649 } } peers { permanent_uuid: "0e32c19034a34c04b2e5f339a9143b69" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 34921 } }
I20260808 16:52:35.544149 21032 raft_consensus.cc:374] T 9ee546be89e54179954fe39c04018cc8 P 7962ba67a970456781409e433168be76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:52:35.544166 21032 raft_consensus.cc:729] T 9ee546be89e54179954fe39c04018cc8 P 7962ba67a970456781409e433168be76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7962ba67a970456781409e433168be76, State: Initialized, Role: FOLLOWER
I20260808 16:52:35.544203 21032 consensus_queue.cc:260] T 9ee546be89e54179954fe39c04018cc8 P 7962ba67a970456781409e433168be76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2771891c9a064fd6b6f4de95834948dc" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 45233 } } peers { permanent_uuid: "7962ba67a970456781409e433168be76" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 37649 } } peers { permanent_uuid: "0e32c19034a34c04b2e5f339a9143b69" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 34921 } }
I20260808 16:52:35.544284 21032 ts_tablet_manager.cc:1428] T 9ee546be89e54179954fe39c04018cc8 P 7962ba67a970456781409e433168be76: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:35.544683 21031 tablet_bootstrap.cc:492] T 9ee546be89e54179954fe39c04018cc8 P 0e32c19034a34c04b2e5f339a9143b69: Bootstrap starting.
I20260808 16:52:35.544982 21031 tablet_bootstrap.cc:654] T 9ee546be89e54179954fe39c04018cc8 P 0e32c19034a34c04b2e5f339a9143b69: Neither blocks nor log segments found. Creating new log.
I20260808 16:52:35.545765 21031 tablet_bootstrap.cc:492] T 9ee546be89e54179954fe39c04018cc8 P 0e32c19034a34c04b2e5f339a9143b69: No bootstrap required, opened a new log
I20260808 16:52:35.545841 21031 ts_tablet_manager.cc:1397] T 9ee546be89e54179954fe39c04018cc8 P 0e32c19034a34c04b2e5f339a9143b69: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 16:52:35.545948 21031 raft_consensus.cc:348] T 9ee546be89e54179954fe39c04018cc8 P 0e32c19034a34c04b2e5f339a9143b69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2771891c9a064fd6b6f4de95834948dc" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 45233 } } peers { permanent_uuid: "7962ba67a970456781409e433168be76" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 37649 } } peers { permanent_uuid: "0e32c19034a34c04b2e5f339a9143b69" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 34921 } }
I20260808 16:52:35.546015 21031 raft_consensus.cc:374] T 9ee546be89e54179954fe39c04018cc8 P 0e32c19034a34c04b2e5f339a9143b69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:52:35.546039 21031 raft_consensus.cc:729] T 9ee546be89e54179954fe39c04018cc8 P 0e32c19034a34c04b2e5f339a9143b69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e32c19034a34c04b2e5f339a9143b69, State: Initialized, Role: FOLLOWER
I20260808 16:52:35.546084 21031 consensus_queue.cc:260] T 9ee546be89e54179954fe39c04018cc8 P 0e32c19034a34c04b2e5f339a9143b69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2771891c9a064fd6b6f4de95834948dc" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 45233 } } peers { permanent_uuid: "7962ba67a970456781409e433168be76" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 37649 } } peers { permanent_uuid: "0e32c19034a34c04b2e5f339a9143b69" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 34921 } }
I20260808 16:52:35.546169 21031 ts_tablet_manager.cc:1428] T 9ee546be89e54179954fe39c04018cc8 P 0e32c19034a34c04b2e5f339a9143b69: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:35.632766 21040 raft_consensus.cc:482] T 9ee546be89e54179954fe39c04018cc8 P 7962ba67a970456781409e433168be76 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 16:52:35.632861 21040 raft_consensus.cc:504] T 9ee546be89e54179954fe39c04018cc8 P 7962ba67a970456781409e433168be76 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2771891c9a064fd6b6f4de95834948dc" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 45233 } } peers { permanent_uuid: "7962ba67a970456781409e433168be76" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 37649 } } peers { permanent_uuid: "0e32c19034a34c04b2e5f339a9143b69" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 34921 } }
I20260808 16:52:35.633147 21040 leader_election.cc:288] T 9ee546be89e54179954fe39c04018cc8 P 7962ba67a970456781409e433168be76 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2771891c9a064fd6b6f4de95834948dc (127.8.24.1:45233), 0e32c19034a34c04b2e5f339a9143b69 (127.8.24.3:34921)
I20260808 16:52:35.636345 21135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:35.633309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:45233 (local address 127.8.24.2:54591)
0808 16:52:35.633419 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:35.633430 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:35.633511 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:35.633682 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:35.633682 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:35.633684 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:52:35.633737 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:35.633739 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:35.634687 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:35.634688 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:35.635575 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:35.635578 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:35.636104 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:35.636106 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:35.636167 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:35.636170 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 16:52:35.636222 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 16:52:35.636258 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93,"mutex_wait_us":48,"thread_start_us":63,"threads_started":1}
I20260808 16:52:35.636345 21029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:35.633370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:54591 (local address 127.8.24.1:45233)
0808 16:52:35.633418 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:35.633420 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:35.633485 (+ 65us) server_negotiation.cc:407] Connection header received
0808 16:52:35.633585 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:35.633586 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:35.633597 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:35.633625 (+ 28us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:52:35.633803 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:35.634568 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:35.635657 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:35.636014 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:35.636074 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:35.636086 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:35.636233 (+ 147us) server_negotiation.cc:299] Negotiation successful
0808 16:52:35.636274 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 16:52:35.636535 20433 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ee546be89e54179954fe39c04018cc8" candidate_uuid: "7962ba67a970456781409e433168be76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2771891c9a064fd6b6f4de95834948dc" is_pre_election: true
I20260808 16:52:35.636616 20433 raft_consensus.cc:2454] T 9ee546be89e54179954fe39c04018cc8 P 2771891c9a064fd6b6f4de95834948dc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7962ba67a970456781409e433168be76 in term 0.
I20260808 16:52:35.636786 20559 leader_election.cc:302] T 9ee546be89e54179954fe39c04018cc8 P 7962ba67a970456781409e433168be76 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2771891c9a064fd6b6f4de95834948dc, 7962ba67a970456781409e433168be76; no voters:
I20260808 16:52:35.636919 21040 raft_consensus.cc:2781] T 9ee546be89e54179954fe39c04018cc8 P 7962ba67a970456781409e433168be76 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 16:52:35.636955 21040 raft_consensus.cc:482] T 9ee546be89e54179954fe39c04018cc8 P 7962ba67a970456781409e433168be76 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 16:52:35.636972 21040 raft_consensus.cc:3037] T 9ee546be89e54179954fe39c04018cc8 P 7962ba67a970456781409e433168be76 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:52:35.637050 21136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:35.633465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.3:34921 (local address 127.8.24.2:47643)
0808 16:52:35.633619 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:35.633629 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:35.633686 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:35.633865 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:35.633865 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:35.633866 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:52:35.634151 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:35.634152 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:35.634901 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:35.634902 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:35.636092 (+ 1190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:35.636095 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:35.636770 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:35.636771 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:35.636840 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:35.636843 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 16:52:35.636923 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 16:52:35.636959 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":83,"threads_started":1}
I20260808 16:52:35.637054 21025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:35.633528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:47643 (local address 127.8.24.3:34921)
0808 16:52:35.633571 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:35.633573 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:35.633689 (+ 116us) server_negotiation.cc:407] Connection header received
0808 16:52:35.633749 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:35.633750 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:35.633763 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:35.633821 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:52:35.634223 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:35.634816 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:35.636171 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:35.636666 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:35.636728 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:35.636739 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:35.636941 (+ 202us) server_negotiation.cc:299] Negotiation successful
0808 16:52:35.636989 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 16:52:35.637218 20908 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ee546be89e54179954fe39c04018cc8" candidate_uuid: "7962ba67a970456781409e433168be76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e32c19034a34c04b2e5f339a9143b69" is_pre_election: true
I20260808 16:52:35.637290 20908 raft_consensus.cc:2454] T 9ee546be89e54179954fe39c04018cc8 P 0e32c19034a34c04b2e5f339a9143b69 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7962ba67a970456781409e433168be76 in term 0.
I20260808 16:52:35.637732 21040 raft_consensus.cc:504] T 9ee546be89e54179954fe39c04018cc8 P 7962ba67a970456781409e433168be76 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2771891c9a064fd6b6f4de95834948dc" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 45233 } } peers { permanent_uuid: "7962ba67a970456781409e433168be76" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 37649 } } peers { permanent_uuid: "0e32c19034a34c04b2e5f339a9143b69" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 34921 } }
I20260808 16:52:35.637835 21040 leader_election.cc:288] T 9ee546be89e54179954fe39c04018cc8 P 7962ba67a970456781409e433168be76 [CANDIDATE]: Term 1 election: Requested vote from peers 2771891c9a064fd6b6f4de95834948dc (127.8.24.1:45233), 0e32c19034a34c04b2e5f339a9143b69 (127.8.24.3:34921)
I20260808 16:52:35.638005 20433 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ee546be89e54179954fe39c04018cc8" candidate_uuid: "7962ba67a970456781409e433168be76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2771891c9a064fd6b6f4de95834948dc"
I20260808 16:52:35.638027 20908 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ee546be89e54179954fe39c04018cc8" candidate_uuid: "7962ba67a970456781409e433168be76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e32c19034a34c04b2e5f339a9143b69"
I20260808 16:52:35.638072 20433 raft_consensus.cc:3037] T 9ee546be89e54179954fe39c04018cc8 P 2771891c9a064fd6b6f4de95834948dc [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:52:35.638091 20908 raft_consensus.cc:3037] T 9ee546be89e54179954fe39c04018cc8 P 0e32c19034a34c04b2e5f339a9143b69 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:52:35.638659 20433 raft_consensus.cc:2454] T 9ee546be89e54179954fe39c04018cc8 P 2771891c9a064fd6b6f4de95834948dc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7962ba67a970456781409e433168be76 in term 1.
I20260808 16:52:35.638967 20908 raft_consensus.cc:2454] T 9ee546be89e54179954fe39c04018cc8 P 0e32c19034a34c04b2e5f339a9143b69 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7962ba67a970456781409e433168be76 in term 1.
I20260808 16:52:35.639146 20559 leader_election.cc:302] T 9ee546be89e54179954fe39c04018cc8 P 7962ba67a970456781409e433168be76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2771891c9a064fd6b6f4de95834948dc, 7962ba67a970456781409e433168be76; no voters:
I20260808 16:52:35.639241 21040 raft_consensus.cc:2781] T 9ee546be89e54179954fe39c04018cc8 P 7962ba67a970456781409e433168be76 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:52:35.639389 21040 raft_consensus.cc:686] T 9ee546be89e54179954fe39c04018cc8 P 7962ba67a970456781409e433168be76 [term 1 LEADER]: Becoming Leader. State: Replica: 7962ba67a970456781409e433168be76, State: Running, Role: LEADER
I20260808 16:52:35.639463 21040 consensus_queue.cc:237] T 9ee546be89e54179954fe39c04018cc8 P 7962ba67a970456781409e433168be76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2771891c9a064fd6b6f4de95834948dc" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 45233 } } peers { permanent_uuid: "7962ba67a970456781409e433168be76" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 37649 } } peers { permanent_uuid: "0e32c19034a34c04b2e5f339a9143b69" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 34921 } }
I20260808 16:52:35.640101 20179 catalog_manager.cc:5495] T 9ee546be89e54179954fe39c04018cc8 P 7962ba67a970456781409e433168be76 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7962ba67a970456781409e433168be76 (127.8.24.2). New cstate: current_term: 1 leader_uuid: "7962ba67a970456781409e433168be76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2771891c9a064fd6b6f4de95834948dc" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 45233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7962ba67a970456781409e433168be76" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 37649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e32c19034a34c04b2e5f339a9143b69" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 34921 } health_report { overall_health: UNKNOWN } } }
I20260808 16:52:35.659976 21027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:35.657666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48906 (local address 127.8.24.2:37649)
0808 16:52:35.657721 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:35.657723 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:35.657738 (+ 15us) server_negotiation.cc:407] Connection header received
0808 16:52:35.657772 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:35.657773 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:35.657794 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:35.657874 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 16:52:35.658005 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:35.658529 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:35.659265 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:35.659455 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:35.659509 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:35.659687 (+ 178us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 16:52:35.659787 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 16:52:35.659855 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:35.659894 (+ 39us) server_negotiation.cc:299] Negotiation successful
0808 16:52:35.659911 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 16:52:35.660771 20433 raft_consensus.cc:1261] T 9ee546be89e54179954fe39c04018cc8 P 2771891c9a064fd6b6f4de95834948dc [term 1 FOLLOWER]: Refusing update from remote peer 7962ba67a970456781409e433168be76: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:52:35.660800 20908 raft_consensus.cc:1261] T 9ee546be89e54179954fe39c04018cc8 P 0e32c19034a34c04b2e5f339a9143b69 [term 1 FOLLOWER]: Refusing update from remote peer 7962ba67a970456781409e433168be76: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:52:35.661013 21137 consensus_queue.cc:1035] T 9ee546be89e54179954fe39c04018cc8 P 7962ba67a970456781409e433168be76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2771891c9a064fd6b6f4de95834948dc" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:52:35.661141 21040 consensus_queue.cc:1035] T 9ee546be89e54179954fe39c04018cc8 P 7962ba67a970456781409e433168be76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e32c19034a34c04b2e5f339a9143b69" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 34921 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:52:35.664378 20179 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55432:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 16:52:35.664465 20179 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55432:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 16:52:35.665082 20179 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e63efa3dec1242f980a4d3123bd3f739: Sending DeleteTablet for 3 replicas of tablet 9ee546be89e54179954fe39c04018cc8
I20260808 16:52:35.665383 20634 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ee546be89e54179954fe39c04018cc8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:52:35 UTC) from {username='oryx', principal='oryx/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:48898
I20260808 16:52:35.665518 21062 tablet_replica.cc:330] T 9ee546be89e54179954fe39c04018cc8 P 7962ba67a970456781409e433168be76: stopping tablet replica
I20260808 16:52:35.665570 21062 raft_consensus.cc:2229] T 9ee546be89e54179954fe39c04018cc8 P 7962ba67a970456781409e433168be76 [term 1 LEADER]: Raft consensus shutting down.
I20260808 16:52:35.665637 21062 raft_consensus.cc:2258] T 9ee546be89e54179954fe39c04018cc8 P 7962ba67a970456781409e433168be76 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:52:35.665596 20179 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55432:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 16:52:35.665704 20873 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ee546be89e54179954fe39c04018cc8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:52:35 UTC) from {username='oryx', principal='oryx/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:41986
W20260808 16:52:35.665746 20179 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 16:52:35.665833 21063 tablet_replica.cc:330] T 9ee546be89e54179954fe39c04018cc8 P 0e32c19034a34c04b2e5f339a9143b69: stopping tablet replica
I20260808 16:52:35.665879 21063 raft_consensus.cc:2229] T 9ee546be89e54179954fe39c04018cc8 P 0e32c19034a34c04b2e5f339a9143b69 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 16:52:35.665913 21063 raft_consensus.cc:2258] T 9ee546be89e54179954fe39c04018cc8 P 0e32c19034a34c04b2e5f339a9143b69 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:52:35.666234 20410 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ee546be89e54179954fe39c04018cc8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:52:35 UTC) from {username='oryx', principal='oryx/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:49408
I20260808 16:52:35.666323 21064 tablet_replica.cc:330] T 9ee546be89e54179954fe39c04018cc8 P 2771891c9a064fd6b6f4de95834948dc: stopping tablet replica
I20260808 16:52:35.666399 21064 raft_consensus.cc:2229] T 9ee546be89e54179954fe39c04018cc8 P 2771891c9a064fd6b6f4de95834948dc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 16:52:35.666443 21064 raft_consensus.cc:2258] T 9ee546be89e54179954fe39c04018cc8 P 2771891c9a064fd6b6f4de95834948dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:52:35.666425 21062 ts_tablet_manager.cc:1905] T 9ee546be89e54179954fe39c04018cc8 P 7962ba67a970456781409e433168be76: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:52:35.666591 21063 ts_tablet_manager.cc:1905] T 9ee546be89e54179954fe39c04018cc8 P 0e32c19034a34c04b2e5f339a9143b69: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:52:35.666877 21064 ts_tablet_manager.cc:1905] T 9ee546be89e54179954fe39c04018cc8 P 2771891c9a064fd6b6f4de95834948dc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:52:35.667851 21063 ts_tablet_manager.cc:1918] T 9ee546be89e54179954fe39c04018cc8 P 0e32c19034a34c04b2e5f339a9143b69: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 16:52:35.667851 21062 ts_tablet_manager.cc:1918] T 9ee546be89e54179954fe39c04018cc8 P 7962ba67a970456781409e433168be76: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 16:52:35.667899 21062 log.cc:1196] T 9ee546be89e54179954fe39c04018cc8 P 7962ba67a970456781409e433168be76: Deleting WAL directory at /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/minicluster-data/ts-1/wal/wals/9ee546be89e54179954fe39c04018cc8
I20260808 16:52:35.667899 21063 log.cc:1196] T 9ee546be89e54179954fe39c04018cc8 P 0e32c19034a34c04b2e5f339a9143b69: Deleting WAL directory at /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/minicluster-data/ts-2/wal/wals/9ee546be89e54179954fe39c04018cc8
I20260808 16:52:35.668092 21062 ts_tablet_manager.cc:1939] T 9ee546be89e54179954fe39c04018cc8 P 7962ba67a970456781409e433168be76: Deleting consensus metadata
I20260808 16:52:35.668092 21063 ts_tablet_manager.cc:1939] T 9ee546be89e54179954fe39c04018cc8 P 0e32c19034a34c04b2e5f339a9143b69: Deleting consensus metadata
I20260808 16:52:35.668392 21064 ts_tablet_manager.cc:1918] T 9ee546be89e54179954fe39c04018cc8 P 2771891c9a064fd6b6f4de95834948dc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 16:52:35.668455 21064 log.cc:1196] T 9ee546be89e54179954fe39c04018cc8 P 2771891c9a064fd6b6f4de95834948dc: Deleting WAL directory at /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/minicluster-data/ts-0/wal/wals/9ee546be89e54179954fe39c04018cc8
I20260808 16:52:35.668659 21064 ts_tablet_manager.cc:1939] T 9ee546be89e54179954fe39c04018cc8 P 2771891c9a064fd6b6f4de95834948dc: Deleting consensus metadata
I20260808 16:52:35.669006 20163 catalog_manager.cc:4841] TS 2771891c9a064fd6b6f4de95834948dc (127.8.24.1:45233): tablet 9ee546be89e54179954fe39c04018cc8 (table test-table [id=38b1314b8f404d8ca14161362f1b2358]) successfully deleted
I20260808 16:52:35.669955 20873 tablet_service.cc:1467] Processing CreateTablet for tablet 3566113b3e394c59a5b66da48c2f40d4 (DEFAULT_TABLE table=test-table [id=f1fc68134c164ef7887842b727ed34d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:52:35.669968 20634 tablet_service.cc:1467] Processing CreateTablet for tablet 3566113b3e394c59a5b66da48c2f40d4 (DEFAULT_TABLE table=test-table [id=f1fc68134c164ef7887842b727ed34d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:52:35.670058 20873 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3566113b3e394c59a5b66da48c2f40d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:52:35.670069 20634 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3566113b3e394c59a5b66da48c2f40d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:52:35.671276 21031 tablet_bootstrap.cc:492] T 3566113b3e394c59a5b66da48c2f40d4 P 0e32c19034a34c04b2e5f339a9143b69: Bootstrap starting.
I20260808 16:52:35.671572 21031 tablet_bootstrap.cc:654] T 3566113b3e394c59a5b66da48c2f40d4 P 0e32c19034a34c04b2e5f339a9143b69: Neither blocks nor log segments found. Creating new log.
I20260808 16:52:35.671819 21032 tablet_bootstrap.cc:492] T 3566113b3e394c59a5b66da48c2f40d4 P 7962ba67a970456781409e433168be76: Bootstrap starting.
I20260808 16:52:35.672216 21032 tablet_bootstrap.cc:654] T 3566113b3e394c59a5b66da48c2f40d4 P 7962ba67a970456781409e433168be76: Neither blocks nor log segments found. Creating new log.
I20260808 16:52:35.672552 20410 tablet_service.cc:1467] Processing CreateTablet for tablet 3566113b3e394c59a5b66da48c2f40d4 (DEFAULT_TABLE table=test-table [id=f1fc68134c164ef7887842b727ed34d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:52:35.672662 20410 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3566113b3e394c59a5b66da48c2f40d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:52:35.673857 21030 tablet_bootstrap.cc:492] T 3566113b3e394c59a5b66da48c2f40d4 P 2771891c9a064fd6b6f4de95834948dc: Bootstrap starting.
I20260808 16:52:35.674151 21030 tablet_bootstrap.cc:654] T 3566113b3e394c59a5b66da48c2f40d4 P 2771891c9a064fd6b6f4de95834948dc: Neither blocks nor log segments found. Creating new log.
I20260808 16:52:35.674327 21031 tablet_bootstrap.cc:492] T 3566113b3e394c59a5b66da48c2f40d4 P 0e32c19034a34c04b2e5f339a9143b69: No bootstrap required, opened a new log
I20260808 16:52:35.674391 21031 ts_tablet_manager.cc:1397] T 3566113b3e394c59a5b66da48c2f40d4 P 0e32c19034a34c04b2e5f339a9143b69: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.000s
I20260808 16:52:35.674492 21031 raft_consensus.cc:348] T 3566113b3e394c59a5b66da48c2f40d4 P 0e32c19034a34c04b2e5f339a9143b69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2771891c9a064fd6b6f4de95834948dc" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 45233 } } peers { permanent_uuid: "7962ba67a970456781409e433168be76" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 37649 } } peers { permanent_uuid: "0e32c19034a34c04b2e5f339a9143b69" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 34921 } }
I20260808 16:52:35.674566 21031 raft_consensus.cc:374] T 3566113b3e394c59a5b66da48c2f40d4 P 0e32c19034a34c04b2e5f339a9143b69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:52:35.674587 21031 raft_consensus.cc:729] T 3566113b3e394c59a5b66da48c2f40d4 P 0e32c19034a34c04b2e5f339a9143b69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e32c19034a34c04b2e5f339a9143b69, State: Initialized, Role: FOLLOWER
I20260808 16:52:35.674633 21031 consensus_queue.cc:260] T 3566113b3e394c59a5b66da48c2f40d4 P 0e32c19034a34c04b2e5f339a9143b69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2771891c9a064fd6b6f4de95834948dc" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 45233 } } peers { permanent_uuid: "7962ba67a970456781409e433168be76" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 37649 } } peers { permanent_uuid: "0e32c19034a34c04b2e5f339a9143b69" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 34921 } }
I20260808 16:52:35.674736 21031 ts_tablet_manager.cc:1428] T 3566113b3e394c59a5b66da48c2f40d4 P 0e32c19034a34c04b2e5f339a9143b69: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:35.675032 21032 tablet_bootstrap.cc:492] T 3566113b3e394c59a5b66da48c2f40d4 P 7962ba67a970456781409e433168be76: No bootstrap required, opened a new log
I20260808 16:52:35.675084 21032 ts_tablet_manager.cc:1397] T 3566113b3e394c59a5b66da48c2f40d4 P 7962ba67a970456781409e433168be76: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260808 16:52:35.675225 21032 raft_consensus.cc:348] T 3566113b3e394c59a5b66da48c2f40d4 P 7962ba67a970456781409e433168be76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2771891c9a064fd6b6f4de95834948dc" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 45233 } } peers { permanent_uuid: "7962ba67a970456781409e433168be76" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 37649 } } peers { permanent_uuid: "0e32c19034a34c04b2e5f339a9143b69" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 34921 } }
I20260808 16:52:35.675294 21032 raft_consensus.cc:374] T 3566113b3e394c59a5b66da48c2f40d4 P 7962ba67a970456781409e433168be76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:52:35.675315 21032 raft_consensus.cc:729] T 3566113b3e394c59a5b66da48c2f40d4 P 7962ba67a970456781409e433168be76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7962ba67a970456781409e433168be76, State: Initialized, Role: FOLLOWER
I20260808 16:52:35.675379 21032 consensus_queue.cc:260] T 3566113b3e394c59a5b66da48c2f40d4 P 7962ba67a970456781409e433168be76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2771891c9a064fd6b6f4de95834948dc" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 45233 } } peers { permanent_uuid: "7962ba67a970456781409e433168be76" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 37649 } } peers { permanent_uuid: "0e32c19034a34c04b2e5f339a9143b69" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 34921 } }
I20260808 16:52:35.675470 21032 ts_tablet_manager.cc:1428] T 3566113b3e394c59a5b66da48c2f40d4 P 7962ba67a970456781409e433168be76: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:35.676256 20163 catalog_manager.cc:4841] TS 0e32c19034a34c04b2e5f339a9143b69 (127.8.24.3:34921): tablet 9ee546be89e54179954fe39c04018cc8 (table test-table [id=38b1314b8f404d8ca14161362f1b2358]) successfully deleted
I20260808 16:52:35.676551 20159 catalog_manager.cc:4841] TS 7962ba67a970456781409e433168be76 (127.8.24.2:37649): tablet 9ee546be89e54179954fe39c04018cc8 (table test-table [id=38b1314b8f404d8ca14161362f1b2358]) successfully deleted
I20260808 16:52:35.676997 21030 tablet_bootstrap.cc:492] T 3566113b3e394c59a5b66da48c2f40d4 P 2771891c9a064fd6b6f4de95834948dc: No bootstrap required, opened a new log
I20260808 16:52:35.677043 21030 ts_tablet_manager.cc:1397] T 3566113b3e394c59a5b66da48c2f40d4 P 2771891c9a064fd6b6f4de95834948dc: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260808 16:52:35.677136 21030 raft_consensus.cc:348] T 3566113b3e394c59a5b66da48c2f40d4 P 2771891c9a064fd6b6f4de95834948dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2771891c9a064fd6b6f4de95834948dc" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 45233 } } peers { permanent_uuid: "7962ba67a970456781409e433168be76" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 37649 } } peers { permanent_uuid: "0e32c19034a34c04b2e5f339a9143b69" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 34921 } }
I20260808 16:52:35.677191 21030 raft_consensus.cc:374] T 3566113b3e394c59a5b66da48c2f40d4 P 2771891c9a064fd6b6f4de95834948dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:52:35.677209 21030 raft_consensus.cc:729] T 3566113b3e394c59a5b66da48c2f40d4 P 2771891c9a064fd6b6f4de95834948dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2771891c9a064fd6b6f4de95834948dc, State: Initialized, Role: FOLLOWER
I20260808 16:52:35.677246 21030 consensus_queue.cc:260] T 3566113b3e394c59a5b66da48c2f40d4 P 2771891c9a064fd6b6f4de95834948dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2771891c9a064fd6b6f4de95834948dc" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 45233 } } peers { permanent_uuid: "7962ba67a970456781409e433168be76" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 37649 } } peers { permanent_uuid: "0e32c19034a34c04b2e5f339a9143b69" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 34921 } }
I20260808 16:52:35.677332 21030 ts_tablet_manager.cc:1428] T 3566113b3e394c59a5b66da48c2f40d4 P 2771891c9a064fd6b6f4de95834948dc: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:35.797935 21137 raft_consensus.cc:482] T 3566113b3e394c59a5b66da48c2f40d4 P 7962ba67a970456781409e433168be76 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 16:52:35.798014 21137 raft_consensus.cc:504] T 3566113b3e394c59a5b66da48c2f40d4 P 7962ba67a970456781409e433168be76 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2771891c9a064fd6b6f4de95834948dc" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 45233 } } peers { permanent_uuid: "7962ba67a970456781409e433168be76" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 37649 } } peers { permanent_uuid: "0e32c19034a34c04b2e5f339a9143b69" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 34921 } }
I20260808 16:52:35.798149 21137 leader_election.cc:288] T 3566113b3e394c59a5b66da48c2f40d4 P 7962ba67a970456781409e433168be76 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2771891c9a064fd6b6f4de95834948dc (127.8.24.1:45233), 0e32c19034a34c04b2e5f339a9143b69 (127.8.24.3:34921)
I20260808 16:52:35.798362 20433 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3566113b3e394c59a5b66da48c2f40d4" candidate_uuid: "7962ba67a970456781409e433168be76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2771891c9a064fd6b6f4de95834948dc" is_pre_election: true
I20260808 16:52:35.798362 20908 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3566113b3e394c59a5b66da48c2f40d4" candidate_uuid: "7962ba67a970456781409e433168be76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e32c19034a34c04b2e5f339a9143b69" is_pre_election: true
I20260808 16:52:35.798466 20433 raft_consensus.cc:2454] T 3566113b3e394c59a5b66da48c2f40d4 P 2771891c9a064fd6b6f4de95834948dc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7962ba67a970456781409e433168be76 in term 0.
I20260808 16:52:35.798466 20908 raft_consensus.cc:2454] T 3566113b3e394c59a5b66da48c2f40d4 P 0e32c19034a34c04b2e5f339a9143b69 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7962ba67a970456781409e433168be76 in term 0.
I20260808 16:52:35.798604 20559 leader_election.cc:302] T 3566113b3e394c59a5b66da48c2f40d4 P 7962ba67a970456781409e433168be76 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e32c19034a34c04b2e5f339a9143b69, 7962ba67a970456781409e433168be76; no voters:
I20260808 16:52:35.798694 21137 raft_consensus.cc:2781] T 3566113b3e394c59a5b66da48c2f40d4 P 7962ba67a970456781409e433168be76 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 16:52:35.798743 21137 raft_consensus.cc:482] T 3566113b3e394c59a5b66da48c2f40d4 P 7962ba67a970456781409e433168be76 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 16:52:35.798768 21137 raft_consensus.cc:3037] T 3566113b3e394c59a5b66da48c2f40d4 P 7962ba67a970456781409e433168be76 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:52:35.799376 21137 raft_consensus.cc:504] T 3566113b3e394c59a5b66da48c2f40d4 P 7962ba67a970456781409e433168be76 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2771891c9a064fd6b6f4de95834948dc" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 45233 } } peers { permanent_uuid: "7962ba67a970456781409e433168be76" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 37649 } } peers { permanent_uuid: "0e32c19034a34c04b2e5f339a9143b69" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 34921 } }
I20260808 16:52:35.799484 21137 leader_election.cc:288] T 3566113b3e394c59a5b66da48c2f40d4 P 7962ba67a970456781409e433168be76 [CANDIDATE]: Term 1 election: Requested vote from peers 2771891c9a064fd6b6f4de95834948dc (127.8.24.1:45233), 0e32c19034a34c04b2e5f339a9143b69 (127.8.24.3:34921)
I20260808 16:52:35.799615 20433 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3566113b3e394c59a5b66da48c2f40d4" candidate_uuid: "7962ba67a970456781409e433168be76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2771891c9a064fd6b6f4de95834948dc"
I20260808 16:52:35.799636 20908 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3566113b3e394c59a5b66da48c2f40d4" candidate_uuid: "7962ba67a970456781409e433168be76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e32c19034a34c04b2e5f339a9143b69"
I20260808 16:52:35.799690 20433 raft_consensus.cc:3037] T 3566113b3e394c59a5b66da48c2f40d4 P 2771891c9a064fd6b6f4de95834948dc [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:52:35.799700 20908 raft_consensus.cc:3037] T 3566113b3e394c59a5b66da48c2f40d4 P 0e32c19034a34c04b2e5f339a9143b69 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:52:35.800248 20433 raft_consensus.cc:2454] T 3566113b3e394c59a5b66da48c2f40d4 P 2771891c9a064fd6b6f4de95834948dc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7962ba67a970456781409e433168be76 in term 1.
I20260808 16:52:35.800295 20908 raft_consensus.cc:2454] T 3566113b3e394c59a5b66da48c2f40d4 P 0e32c19034a34c04b2e5f339a9143b69 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7962ba67a970456781409e433168be76 in term 1.
I20260808 16:52:35.800427 20559 leader_election.cc:302] T 3566113b3e394c59a5b66da48c2f40d4 P 7962ba67a970456781409e433168be76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2771891c9a064fd6b6f4de95834948dc, 7962ba67a970456781409e433168be76; no voters:
I20260808 16:52:35.800520 21137 raft_consensus.cc:2781] T 3566113b3e394c59a5b66da48c2f40d4 P 7962ba67a970456781409e433168be76 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:52:35.800567 21137 raft_consensus.cc:686] T 3566113b3e394c59a5b66da48c2f40d4 P 7962ba67a970456781409e433168be76 [term 1 LEADER]: Becoming Leader. State: Replica: 7962ba67a970456781409e433168be76, State: Running, Role: LEADER
I20260808 16:52:35.800624 21137 consensus_queue.cc:237] T 3566113b3e394c59a5b66da48c2f40d4 P 7962ba67a970456781409e433168be76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2771891c9a064fd6b6f4de95834948dc" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 45233 } } peers { permanent_uuid: "7962ba67a970456781409e433168be76" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 37649 } } peers { permanent_uuid: "0e32c19034a34c04b2e5f339a9143b69" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 34921 } }
I20260808 16:52:35.801144 20181 catalog_manager.cc:5495] T 3566113b3e394c59a5b66da48c2f40d4 P 7962ba67a970456781409e433168be76 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7962ba67a970456781409e433168be76 (127.8.24.2). New cstate: current_term: 1 leader_uuid: "7962ba67a970456781409e433168be76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2771891c9a064fd6b6f4de95834948dc" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 45233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7962ba67a970456781409e433168be76" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 37649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e32c19034a34c04b2e5f339a9143b69" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 34921 } health_report { overall_health: UNKNOWN } } }
I20260808 16:52:35.823729 21029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:35.821786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:50715 (local address 127.8.24.1:45233)
0808 16:52:35.821848 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:35.821850 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:35.821912 (+ 62us) server_negotiation.cc:407] Connection header received
0808 16:52:35.821960 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:35.821961 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:35.821970 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:35.822011 (+ 41us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 16:52:35.822086 (+ 75us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:35.822594 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:35.823259 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:35.823430 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:35.823470 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:35.823577 (+ 107us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 16:52:35.823600 (+ 23us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 16:52:35.823630 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:35.823659 (+ 29us) server_negotiation.cc:299] Negotiation successful
0808 16:52:35.823682 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 16:52:35.823740 21160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:35.821730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:45233 (local address 127.8.24.2:50715)
0808 16:52:35.821859 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:35.821869 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:35.821922 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:35.822012 (+ 90us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:35.822012 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:35.822014 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 16:52:35.822053 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:35.822054 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:35.822647 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:35.822648 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:35.823204 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:35.823205 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:35.823479 (+ 274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:35.823479 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:35.823513 (+ 34us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:35.823525 (+ 12us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 16:52:35.823630 (+ 105us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 16:52:35.823631 (+ 1us) client_negotiation.cc:769] Sending connection context
0808 16:52:35.823657 (+ 26us) client_negotiation.cc:240] Negotiation successful
0808 16:52:35.823689 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":61,"threads_started":1}
I20260808 16:52:35.825409 20908 raft_consensus.cc:1261] T 3566113b3e394c59a5b66da48c2f40d4 P 0e32c19034a34c04b2e5f339a9143b69 [term 1 FOLLOWER]: Refusing update from remote peer 7962ba67a970456781409e433168be76: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:52:35.825440 20433 raft_consensus.cc:1261] T 3566113b3e394c59a5b66da48c2f40d4 P 2771891c9a064fd6b6f4de95834948dc [term 1 FOLLOWER]: Refusing update from remote peer 7962ba67a970456781409e433168be76: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:52:35.825587 21146 consensus_queue.cc:1035] T 3566113b3e394c59a5b66da48c2f40d4 P 7962ba67a970456781409e433168be76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e32c19034a34c04b2e5f339a9143b69" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 34921 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:52:35.825676 21137 consensus_queue.cc:1035] T 3566113b3e394c59a5b66da48c2f40d4 P 7962ba67a970456781409e433168be76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2771891c9a064fd6b6f4de95834948dc" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:52:35.831667 21027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:35.829614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:34449 (local address 127.8.24.2:37649)
0808 16:52:35.829656 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:35.829658 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:35.829666 (+ 8us) server_negotiation.cc:407] Connection header received
0808 16:52:35.829691 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:35.829692 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:35.829701 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:35.829751 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 16:52:35.829830 (+ 79us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:35.830260 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:35.830975 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:35.831231 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:35.831289 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:35.831437 (+ 148us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 16:52:35.831469 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 16:52:35.831511 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:35.831585 (+ 74us) server_negotiation.cc:299] Negotiation successful
0808 16:52:35.831608 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 16:52:35.831678 21106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:35.829571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.2:37649 (local address 127.8.24.1:34449)
0808 16:52:35.829614 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:35.829623 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:35.829656 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:35.829751 (+ 95us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:35.829752 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:35.829753 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 16:52:35.829788 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:35.829789 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:35.830308 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:35.830309 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:35.830920 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:35.830922 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:35.831299 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:35.831300 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:35.831358 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:35.831365 (+ 7us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 16:52:35.831533 (+ 168us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 16:52:35.831534 (+ 1us) client_negotiation.cc:769] Sending connection context
0808 16:52:35.831576 (+ 42us) client_negotiation.cc:240] Negotiation successful
0808 16:52:35.831625 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23}
I20260808 16:52:35.837000 20181 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55432:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 16:52:35.837083 20181 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55432:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 16:52:35.837604 20181 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e63efa3dec1242f980a4d3123bd3f739: Sending DeleteTablet for 3 replicas of tablet 3566113b3e394c59a5b66da48c2f40d4
I20260808 16:52:35.837821 20634 tablet_service.cc:1514] Processing DeleteTablet for tablet 3566113b3e394c59a5b66da48c2f40d4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:52:35 UTC) from {username='oryx', principal='oryx/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:48898
I20260808 16:52:35.837827 20410 tablet_service.cc:1514] Processing DeleteTablet for tablet 3566113b3e394c59a5b66da48c2f40d4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:52:35 UTC) from {username='oryx', principal='oryx/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:49408
I20260808 16:52:35.837826 20873 tablet_service.cc:1514] Processing DeleteTablet for tablet 3566113b3e394c59a5b66da48c2f40d4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:52:35 UTC) from {username='oryx', principal='oryx/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:41986
I20260808 16:52:35.837905 21064 tablet_replica.cc:330] T 3566113b3e394c59a5b66da48c2f40d4 P 2771891c9a064fd6b6f4de95834948dc: stopping tablet replica
I20260808 16:52:35.837905 21063 tablet_replica.cc:330] T 3566113b3e394c59a5b66da48c2f40d4 P 0e32c19034a34c04b2e5f339a9143b69: stopping tablet replica
I20260808 16:52:35.837906 21062 tablet_replica.cc:330] T 3566113b3e394c59a5b66da48c2f40d4 P 7962ba67a970456781409e433168be76: stopping tablet replica
I20260808 16:52:35.837949 21062 raft_consensus.cc:2229] T 3566113b3e394c59a5b66da48c2f40d4 P 7962ba67a970456781409e433168be76 [term 1 LEADER]: Raft consensus shutting down.
I20260808 16:52:35.837949 21064 raft_consensus.cc:2229] T 3566113b3e394c59a5b66da48c2f40d4 P 2771891c9a064fd6b6f4de95834948dc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 16:52:35.837955 21063 raft_consensus.cc:2229] T 3566113b3e394c59a5b66da48c2f40d4 P 0e32c19034a34c04b2e5f339a9143b69 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 16:52:35.837982 21063 raft_consensus.cc:2258] T 3566113b3e394c59a5b66da48c2f40d4 P 0e32c19034a34c04b2e5f339a9143b69 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:52:35.837982 21064 raft_consensus.cc:2258] T 3566113b3e394c59a5b66da48c2f40d4 P 2771891c9a064fd6b6f4de95834948dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:52:35.838004 21062 raft_consensus.cc:2258] T 3566113b3e394c59a5b66da48c2f40d4 P 7962ba67a970456781409e433168be76 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:52:35.838268 21062 ts_tablet_manager.cc:1905] T 3566113b3e394c59a5b66da48c2f40d4 P 7962ba67a970456781409e433168be76: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:52:35.838279 21064 ts_tablet_manager.cc:1905] T 3566113b3e394c59a5b66da48c2f40d4 P 2771891c9a064fd6b6f4de95834948dc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:52:35.838294 21063 ts_tablet_manager.cc:1905] T 3566113b3e394c59a5b66da48c2f40d4 P 0e32c19034a34c04b2e5f339a9143b69: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:52:35.839527 21062 ts_tablet_manager.cc:1918] T 3566113b3e394c59a5b66da48c2f40d4 P 7962ba67a970456781409e433168be76: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 16:52:35.839576 21062 log.cc:1196] T 3566113b3e394c59a5b66da48c2f40d4 P 7962ba67a970456781409e433168be76: Deleting WAL directory at /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/minicluster-data/ts-1/wal/wals/3566113b3e394c59a5b66da48c2f40d4
I20260808 16:52:35.839735 21063 ts_tablet_manager.cc:1918] T 3566113b3e394c59a5b66da48c2f40d4 P 0e32c19034a34c04b2e5f339a9143b69: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 16:52:35.839775 21062 ts_tablet_manager.cc:1939] T 3566113b3e394c59a5b66da48c2f40d4 P 7962ba67a970456781409e433168be76: Deleting consensus metadata
I20260808 16:52:35.839797 21063 log.cc:1196] T 3566113b3e394c59a5b66da48c2f40d4 P 0e32c19034a34c04b2e5f339a9143b69: Deleting WAL directory at /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/minicluster-data/ts-2/wal/wals/3566113b3e394c59a5b66da48c2f40d4
I20260808 16:52:35.839896 21064 ts_tablet_manager.cc:1918] T 3566113b3e394c59a5b66da48c2f40d4 P 2771891c9a064fd6b6f4de95834948dc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 16:52:35.839949 21064 log.cc:1196] T 3566113b3e394c59a5b66da48c2f40d4 P 2771891c9a064fd6b6f4de95834948dc: Deleting WAL directory at /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786207919384338-8288-0/minicluster-data/ts-0/wal/wals/3566113b3e394c59a5b66da48c2f40d4
I20260808 16:52:35.840008 21063 ts_tablet_manager.cc:1939] T 3566113b3e394c59a5b66da48c2f40d4 P 0e32c19034a34c04b2e5f339a9143b69: Deleting consensus metadata
I20260808 16:52:35.840050 20159 catalog_manager.cc:4841] TS 7962ba67a970456781409e433168be76 (127.8.24.2:37649): tablet 3566113b3e394c59a5b66da48c2f40d4 (table test-table [id=f1fc68134c164ef7887842b727ed34d2]) successfully deleted
I20260808 16:52:35.840134 21064 ts_tablet_manager.cc:1939] T 3566113b3e394c59a5b66da48c2f40d4 P 2771891c9a064fd6b6f4de95834948dc: Deleting consensus metadata
I20260808 16:52:35.840261 20163 catalog_manager.cc:4841] TS 0e32c19034a34c04b2e5f339a9143b69 (127.8.24.3:34921): tablet 3566113b3e394c59a5b66da48c2f40d4 (table test-table [id=f1fc68134c164ef7887842b727ed34d2]) successfully deleted
I20260808 16:52:35.840407 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 20296
W20260808 16:52:35.858419 20163 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.24.1:45233 (error 108)
I20260808 16:52:35.858587 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 20534
I20260808 16:52:35.864034 21164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:35.860984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:45233 (local address 127.0.0.1:38628)
0808 16:52:35.863952 (+ 2968us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:35.863998 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.1:45233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2950,"thread_start_us":1483,"threads_started":1}
W20260808 16:52:35.864107 20163 catalog_manager.cc:4584] TS 2771891c9a064fd6b6f4de95834948dc (127.8.24.1:45233): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3566113b3e394c59a5b66da48c2f40d4: Network error: Client connection negotiation failed: client connection to 127.8.24.1:45233: connect: Connection refused (error 111)
I20260808 16:52:35.870819 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 20756
I20260808 16:52:35.883052 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 20135
2026-08-08T16:52:35Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (2999 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786207919384338-8288-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 16:52:35 dist-test-slave-6zbq krb5kdc[21169](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 16:52:35 dist-test-slave-6zbq krb5kdc[21169](info): set up 2 sockets
Aug 08 16:52:35 dist-test-slave-6zbq krb5kdc[21169](info): commencing operation
krb5kdc: starting...
W20260808 16:52:37.008467 8288 mini_kdc.cc:121] Time spent starting KDC: real 1.101s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786207919384338-8288-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786207919384338-8288-0/krb5kdc/test-user.keytab.
Aug 08 16:52:37 dist-test-slave-6zbq krb5kdc[21169](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207957, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T16:52:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T16:52:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.24.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
I20260808 16:52:37.113044 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786207919384338-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786207919384338-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786207919384338-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:32891
--webserver_interface=127.8.24.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:36539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:32891
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:52:37.190179 21193 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:52:37.190375 21193 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:52:37.190407 21193 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:52:37.191512 21193 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 16:52:37.191550 21193 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:52:37.191568 21193 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:52:37.191584 21193 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 16:52:37.191599 21193 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 16:52:37.193035 21193 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:36539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786207919384338-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786207919384338-8288-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:32891
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:32891
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.24.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21193
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786207919384338-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:52:37.193264 21193 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:52:37.193450 21193 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:52:37.197821 21202 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:37.197815 21204 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:37.197849 21193 server_base.cc:1034] running on GCE node
W20260808 16:52:37.197885 21200 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:37.198243 21193 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:52:37.198534 21193 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:52:37.199666 21193 hybrid_clock.cc:648] HybridClock initialized: now 1786207957199650 us; error 32 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-08T16:52:37Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1322 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 16:52:37 dist-test-slave-6zbq krb5kdc[21210](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 16:52:37 dist-test-slave-6zbq krb5kdc[21210](info): set up 2 sockets
Aug 08 16:52:37 dist-test-slave-6zbq krb5kdc[21210](info): commencing operation
krb5kdc: starting...
W20260808 16:52:38.333043 8288 mini_kdc.cc:121] Time spent starting KDC: real 1.104s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/krb5kdc/test-user.keytab.
Aug 08 16:52:38 dist-test-slave-6zbq krb5kdc[21210](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207958, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T16:52:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T16:52:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.24.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
I20260808 16:52:38.438702 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:43045
--webserver_interface=127.8.24.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:43365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:43045
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:52:38.516570 21227 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:52:38.516762 21227 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:52:38.516793 21227 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:52:38.517874 21227 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 16:52:38.517912 21227 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:52:38.517931 21227 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:52:38.517946 21227 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 16:52:38.517963 21227 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 16:52:38.519416 21227 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:43365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:43045
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:43045
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.24.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21227
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:52:38.519652 21227 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:52:38.519838 21227 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:52:38.524251 21241 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:38.524317 21237 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:38.524257 21236 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:38.524463 21227 server_base.cc:1034] running on GCE node
I20260808 16:52:38.524770 21227 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:52:38.525022 21227 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:52:38.526160 21227 hybrid_clock.cc:648] HybridClock initialized: now 1786207958526141 us; error 36 us; skew 500 ppm
Aug 08 16:52:38 dist-test-slave-6zbq krb5kdc[21210](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207958, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:52:38.531477 21227 init.cc:377] Logged in from keytab as kudu/127.8.24.62@KRBTEST.COM (short username kudu)
I20260808 16:52:38.531886 21227 webserver.cc:460] Webserver started at http://127.8.24.62:42887/ using document root <none> and password file <none>
I20260808 16:52:38.532081 21227 fs_manager.cc:359] Metadata directory not provided
I20260808 16:52:38.532126 21227 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:52:38.532253 21227 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:52:38.533183 21227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/master-0/data/instance:
uuid: "f6eacb891fa04c52b99587ac9f7c0587"
format_stamp: "Formatted at 2026-08-08 16:52:38 on dist-test-slave-6zbq"
server_key: "caa3eac9b2669beb2db8e221d9db264c"
server_key_iv: "ecce497d48d834b54482c958cafbc79c"
server_key_version: "encryptionkey@0"
I20260808 16:52:38.533506 21227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/master-0/wal/instance:
uuid: "f6eacb891fa04c52b99587ac9f7c0587"
format_stamp: "Formatted at 2026-08-08 16:52:38 on dist-test-slave-6zbq"
server_key: "caa3eac9b2669beb2db8e221d9db264c"
server_key_iv: "ecce497d48d834b54482c958cafbc79c"
server_key_version: "encryptionkey@0"
I20260808 16:52:38.534724 21227 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 16:52:38.535454 21253 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:38.535653 21227 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 16:52:38.535741 21227 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/master-0/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/master-0/wal
uuid: "f6eacb891fa04c52b99587ac9f7c0587"
format_stamp: "Formatted at 2026-08-08 16:52:38 on dist-test-slave-6zbq"
server_key: "caa3eac9b2669beb2db8e221d9db264c"
server_key_iv: "ecce497d48d834b54482c958cafbc79c"
server_key_version: "encryptionkey@0"
I20260808 16:52:38.535811 21227 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:52:38.549597 21227 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:52:38.551502 21227 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:52:38.551623 21227 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:52:38.555174 21227 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.62:43045
I20260808 16:52:38.555212 21334 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.62:43045 every 8 connection(s)
I20260808 16:52:38.555496 21227 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
I20260808 16:52:38.556237 21335 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:52:38.558233 21335 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6eacb891fa04c52b99587ac9f7c0587: Bootstrap starting.
I20260808 16:52:38.558776 21335 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f6eacb891fa04c52b99587ac9f7c0587: Neither blocks nor log segments found. Creating new log.
I20260808 16:52:38.559027 21335 log.cc:824] T 00000000000000000000000000000000 P f6eacb891fa04c52b99587ac9f7c0587: Log is configured to *not* fsync() on all Append() calls
I20260808 16:52:38.559664 21335 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6eacb891fa04c52b99587ac9f7c0587: No bootstrap required, opened a new log
I20260808 16:52:38.561182 21335 raft_consensus.cc:348] T 00000000000000000000000000000000 P f6eacb891fa04c52b99587ac9f7c0587 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6eacb891fa04c52b99587ac9f7c0587" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 43045 } }
I20260808 16:52:38.561305 21335 raft_consensus.cc:374] T 00000000000000000000000000000000 P f6eacb891fa04c52b99587ac9f7c0587 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:52:38.561335 21335 raft_consensus.cc:729] T 00000000000000000000000000000000 P f6eacb891fa04c52b99587ac9f7c0587 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6eacb891fa04c52b99587ac9f7c0587, State: Initialized, Role: FOLLOWER
I20260808 16:52:38.561465 21335 consensus_queue.cc:260] T 00000000000000000000000000000000 P f6eacb891fa04c52b99587ac9f7c0587 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6eacb891fa04c52b99587ac9f7c0587" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 43045 } }
I20260808 16:52:38.561522 21335 raft_consensus.cc:388] T 00000000000000000000000000000000 P f6eacb891fa04c52b99587ac9f7c0587 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 16:52:38.561543 21335 raft_consensus.cc:482] T 00000000000000000000000000000000 P f6eacb891fa04c52b99587ac9f7c0587 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 16:52:38.561587 21335 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f6eacb891fa04c52b99587ac9f7c0587 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:52:38.562093 21335 raft_consensus.cc:504] T 00000000000000000000000000000000 P f6eacb891fa04c52b99587ac9f7c0587 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6eacb891fa04c52b99587ac9f7c0587" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 43045 } }
I20260808 16:52:38.562191 21335 leader_election.cc:302] T 00000000000000000000000000000000 P f6eacb891fa04c52b99587ac9f7c0587 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f6eacb891fa04c52b99587ac9f7c0587; no voters:
I20260808 16:52:38.562362 21335 leader_election.cc:288] T 00000000000000000000000000000000 P f6eacb891fa04c52b99587ac9f7c0587 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 16:52:38.562410 21338 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f6eacb891fa04c52b99587ac9f7c0587 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:52:38.562563 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 21227
I20260808 16:52:38.562592 21335 sys_catalog.cc:564] T 00000000000000000000000000000000 P f6eacb891fa04c52b99587ac9f7c0587 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 16:52:38.562690 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/master-0/wal/instance
I20260808 16:52:38.562773 21338 raft_consensus.cc:686] T 00000000000000000000000000000000 P f6eacb891fa04c52b99587ac9f7c0587 [term 1 LEADER]: Becoming Leader. State: Replica: f6eacb891fa04c52b99587ac9f7c0587, State: Running, Role: LEADER
I20260808 16:52:38.562881 8288 external_mini_cluster.cc:1428] Setting key e089c0e3984cb1c10792c80bf3f10c66
I20260808 16:52:38.562868 21338 consensus_queue.cc:237] T 00000000000000000000000000000000 P f6eacb891fa04c52b99587ac9f7c0587 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6eacb891fa04c52b99587ac9f7c0587" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 43045 } }
I20260808 16:52:38.563606 21339 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6eacb891fa04c52b99587ac9f7c0587 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f6eacb891fa04c52b99587ac9f7c0587" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6eacb891fa04c52b99587ac9f7c0587" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 43045 } } }
I20260808 16:52:38.563694 21339 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6eacb891fa04c52b99587ac9f7c0587 [sys.catalog]: This master's current role is: LEADER
I20260808 16:52:38.564105 21340 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6eacb891fa04c52b99587ac9f7c0587 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f6eacb891fa04c52b99587ac9f7c0587. Latest consensus state: current_term: 1 leader_uuid: "f6eacb891fa04c52b99587ac9f7c0587" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6eacb891fa04c52b99587ac9f7c0587" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 43045 } } }
I20260808 16:52:38.564373 21340 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6eacb891fa04c52b99587ac9f7c0587 [sys.catalog]: This master's current role is: LEADER
W20260808 16:52:38.564491 21365 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f6eacb891fa04c52b99587ac9f7c0587: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 16:52:38.564559 21365 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 16:52:38.564620 21364 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 16:52:38.564765 21364 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 16:52:38.566278 21364 catalog_manager.cc:1347] Generated new cluster ID: 65134025533c43eeb8d8a171556cbbae
I20260808 16:52:38.566327 21364 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 16:52:38 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207958, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260808 16:52:38.572695 21354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:38.563347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32862 (local address 127.8.24.62:43045)
0808 16:52:38.563973 (+ 626us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:38.563981 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:38.564012 (+ 31us) server_negotiation.cc:407] Connection header received
0808 16:52:38.564159 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:38.564164 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:38.564208 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:38.564421 (+ 213us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:38.565109 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:38.566245 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:38.566990 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:38.567402 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:38.567492 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:38.569476 (+ 1984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:38.569479 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:38.569480 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:38.569491 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:38.571370 (+ 1879us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:38.571674 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:38.571676 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:38.571676 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:38.571759 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:38.571930 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:38.571931 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:38.571931 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:38.572136 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:38.572268 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:38.572383 (+ 115us) server_negotiation.cc:299] Negotiation successful
0808 16:52:38.572509 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":585,"thread_start_us":58,"threads_started":1}
I20260808 16:52:38.573207 21364 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 16:52:38.573889 21364 catalog_manager.cc:1504] Loading token signing keys...
I20260808 16:52:38.580202 21364 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f6eacb891fa04c52b99587ac9f7c0587: Generated new TSK 0
I20260808 16:52:38.580353 21364 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.24.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.1.keytab.
Entry for principal kudu/127.8.24.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.1.keytab.
I20260808 16:52:38.609411 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.1:0
--local_ip_for_outbound_sockets=127.8.24.1
--webserver_interface=127.8.24.1
--webserver_port=0
--tserver_master_addrs=127.8.24.62:43045
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.1.keytab
--principal=kudu/127.8.24.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:43365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:52:38.723129 21376 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:52:38.723366 21376 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:52:38.723395 21376 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:52:38.725013 21376 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:52:38.725056 21376 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:52:38.725092 21376 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.1
I20260808 16:52:38.727082 21376 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:43365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.1.keytab
--principal=kudu/127.8.24.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.24.1
--webserver_port=0
--tserver_master_addrs=127.8.24.62:43045
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21376
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.1
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:52:38.727291 21376 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:52:38.727504 21376 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:52:38.732352 21387 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:38.732376 21388 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:38.732383 21390 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:38.732520 21376 server_base.cc:1034] running on GCE node
I20260808 16:52:38.732739 21376 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:52:38.732959 21376 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:52:38.734097 21376 hybrid_clock.cc:648] HybridClock initialized: now 1786207958734084 us; error 27 us; skew 500 ppm
Aug 08 16:52:38 dist-test-slave-6zbq krb5kdc[21210](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207958, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:52:38.736290 21376 init.cc:377] Logged in from keytab as kudu/127.8.24.1@KRBTEST.COM (short username kudu)
I20260808 16:52:38.736694 21376 webserver.cc:460] Webserver started at http://127.8.24.1:46041/ using document root <none> and password file <none>
I20260808 16:52:38.736878 21376 fs_manager.cc:359] Metadata directory not provided
I20260808 16:52:38.736923 21376 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:52:38.737042 21376 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:52:38.737973 21376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/ts-0/data/instance:
uuid: "4e8635a62a7c43b4b314aedf778f9eb0"
format_stamp: "Formatted at 2026-08-08 16:52:38 on dist-test-slave-6zbq"
server_key: "45adf9628f2466ab5a4da8cc429998b9"
server_key_iv: "a3ef720e0fa7aecebea922eb2c6bca14"
server_key_version: "encryptionkey@0"
I20260808 16:52:38.738288 21376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/ts-0/wal/instance:
uuid: "4e8635a62a7c43b4b314aedf778f9eb0"
format_stamp: "Formatted at 2026-08-08 16:52:38 on dist-test-slave-6zbq"
server_key: "45adf9628f2466ab5a4da8cc429998b9"
server_key_iv: "a3ef720e0fa7aecebea922eb2c6bca14"
server_key_version: "encryptionkey@0"
I20260808 16:52:38.739500 21376 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 16:52:38.740150 21401 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:38.740291 21376 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:38.740371 21376 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/ts-0/wal
uuid: "4e8635a62a7c43b4b314aedf778f9eb0"
format_stamp: "Formatted at 2026-08-08 16:52:38 on dist-test-slave-6zbq"
server_key: "45adf9628f2466ab5a4da8cc429998b9"
server_key_iv: "a3ef720e0fa7aecebea922eb2c6bca14"
server_key_version: "encryptionkey@0"
I20260808 16:52:38.740434 21376 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:52:38.766232 21376 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:52:38.767771 21376 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:52:38.767889 21376 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:52:38.768075 21376 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:52:38.768332 21376 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:52:38.768363 21376 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:38.768389 21376 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:52:38.768407 21376 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:38.773639 21376 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.1:43545
I20260808 16:52:38.773671 21571 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.1:43545 every 8 connection(s)
I20260808 16:52:38.773926 21376 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/ts-0/data/info.pb
Aug 08 16:52:38 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207958, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.1@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260808 16:52:38.781411 21354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:38.774561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:38745 (local address 127.8.24.62:43045)
0808 16:52:38.774630 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:38.774632 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:38.774768 (+ 136us) server_negotiation.cc:407] Connection header received
0808 16:52:38.775172 (+ 404us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:38.775174 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:38.775185 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:38.775223 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:38.776004 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:38.776475 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:38.777326 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:38.777502 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:38.777543 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:38.779359 (+ 1816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:38.779360 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:38.779361 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:38.779366 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:38.780380 (+ 1014us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:38.780694 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:38.780694 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:38.780695 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:38.780738 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:38.780908 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:38.780909 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:38.780909 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:38.780994 (+ 85us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:38.781037 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:38.781276 (+ 239us) server_negotiation.cc:299] Negotiation successful
0808 16:52:38.781349 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 16:52:38.781461 21578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:38.774540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:43045 (local address 127.8.24.1:38745)
0808 16:52:38.774672 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:38.774703 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:38.775094 (+ 391us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:38.775271 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:38.775274 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:38.775767 (+ 493us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:38.775941 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:38.775945 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:38.776532 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:38.776532 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:38.777266 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:38.777268 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:38.777553 (+ 285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:38.777553 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:38.777640 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:38.778051 (+ 411us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:38.778059 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:38.779288 (+ 1229us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:52:38.780443 (+ 1155us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:38.780448 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:38.780450 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:38.780638 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:38.780781 (+ 143us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:38.780781 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:38.780782 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:38.780869 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:38.781034 (+ 165us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:52:38.781036 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:52:38.781142 (+ 106us) client_negotiation.cc:769] Sending connection context
0808 16:52:38.781215 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 16:52:38.781322 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":73,"threads_started":1}
I20260808 16:52:38.781881 21573 heartbeater.cc:347] Connected to a master server at 127.8.24.62:43045
I20260808 16:52:38.781970 21573 heartbeater.cc:464] Registering TS with master...
I20260808 16:52:38.782126 21573 heartbeater.cc:511] Master 127.8.24.62:43045 requested a full tablet report, sending...
I20260808 16:52:38.782591 21275 ts_manager.cc:194] Registered new tserver with Master: 4e8635a62a7c43b4b314aedf778f9eb0 (127.8.24.1:43545)
I20260808 16:52:38.783142 21275 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.1@KRBTEST.COM'} at 127.8.24.1:38745
I20260808 16:52:38.784063 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 21376
I20260808 16:52:38.784142 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/ts-0/wal/instance
I20260808 16:52:38.784271 8288 external_mini_cluster.cc:1428] Setting key 6f87d348a50e4c81706782e668b3b293
WARNING: no policy specified for kudu/127.8.24.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.2.keytab.
Entry for principal kudu/127.8.24.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.2.keytab.
I20260808 16:52:38.805390 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.2:0
--local_ip_for_outbound_sockets=127.8.24.2
--webserver_interface=127.8.24.2
--webserver_port=0
--tserver_master_addrs=127.8.24.62:43045
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.2.keytab
--principal=kudu/127.8.24.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:43365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:52:38.883252 21581 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:52:38.883430 21581 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:52:38.883450 21581 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:52:38.884539 21581 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:52:38.884577 21581 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:52:38.884603 21581 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.2
I20260808 16:52:38.885977 21581 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:43365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.2.keytab
--principal=kudu/127.8.24.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.24.2
--webserver_port=0
--tserver_master_addrs=127.8.24.62:43045
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21581
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.2
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:52:38.886190 21581 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:52:38.886422 21581 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:52:38.890915 21589 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:38.890950 21586 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:38.890925 21587 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:38.891031 21581 server_base.cc:1034] running on GCE node
I20260808 16:52:38.891300 21581 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:52:38.891508 21581 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:52:38.892637 21581 hybrid_clock.cc:648] HybridClock initialized: now 1786207958892626 us; error 27 us; skew 500 ppm
Aug 08 16:52:38 dist-test-slave-6zbq krb5kdc[21210](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207958, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:52:38.894869 21581 init.cc:377] Logged in from keytab as kudu/127.8.24.2@KRBTEST.COM (short username kudu)
I20260808 16:52:38.895306 21581 webserver.cc:460] Webserver started at http://127.8.24.2:37195/ using document root <none> and password file <none>
I20260808 16:52:38.895517 21581 fs_manager.cc:359] Metadata directory not provided
I20260808 16:52:38.895560 21581 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:52:38.895682 21581 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:52:38.896629 21581 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/ts-1/data/instance:
uuid: "c4a38b53515040d4852c2933315091a6"
format_stamp: "Formatted at 2026-08-08 16:52:38 on dist-test-slave-6zbq"
server_key: "c19c62f5bdb506be148de2a5dd391227"
server_key_iv: "7d915a357c612752c27ed4c4089e277b"
server_key_version: "encryptionkey@0"
I20260808 16:52:38.896937 21581 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/ts-1/wal/instance:
uuid: "c4a38b53515040d4852c2933315091a6"
format_stamp: "Formatted at 2026-08-08 16:52:38 on dist-test-slave-6zbq"
server_key: "c19c62f5bdb506be148de2a5dd391227"
server_key_iv: "7d915a357c612752c27ed4c4089e277b"
server_key_version: "encryptionkey@0"
I20260808 16:52:38.898089 21581 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 16:52:38.898823 21597 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:38.898972 21581 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:38.899053 21581 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/ts-1/wal
uuid: "c4a38b53515040d4852c2933315091a6"
format_stamp: "Formatted at 2026-08-08 16:52:38 on dist-test-slave-6zbq"
server_key: "c19c62f5bdb506be148de2a5dd391227"
server_key_iv: "7d915a357c612752c27ed4c4089e277b"
server_key_version: "encryptionkey@0"
I20260808 16:52:38.899133 21581 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:52:38.914006 21581 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:52:38.915624 21581 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:52:38.915752 21581 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:52:38.915939 21581 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:52:38.916219 21581 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:52:38.916252 21581 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:38.916278 21581 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:52:38.916296 21581 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:38.921636 21581 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.2:43333
I20260808 16:52:38.921669 21779 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.2:43333 every 8 connection(s)
I20260808 16:52:38.921931 21581 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/ts-1/data/info.pb
Aug 08 16:52:38 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207958, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.2@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260808 16:52:38.929404 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 21581
I20260808 16:52:38.929503 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/ts-1/wal/instance
I20260808 16:52:38.929708 8288 external_mini_cluster.cc:1428] Setting key ebb648df979f2c943ea7c88ff713380d
I20260808 16:52:38.930101 21354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:38.922575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:52935 (local address 127.8.24.62:43045)
0808 16:52:38.922635 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:38.922637 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:38.922793 (+ 156us) server_negotiation.cc:407] Connection header received
0808 16:52:38.923203 (+ 410us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:38.923204 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:38.923214 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:38.923260 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:38.924387 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:38.924875 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:38.925760 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:38.925938 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:38.925978 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:38.927745 (+ 1767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:38.927746 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:38.927747 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:38.927753 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:38.928915 (+ 1162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:38.929223 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:38.929224 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:38.929225 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:38.929287 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:38.929503 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:38.929504 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:38.929504 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:38.929632 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:38.929698 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:38.929939 (+ 241us) server_negotiation.cc:299] Negotiation successful
0808 16:52:38.930013 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 16:52:38.930373 21785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:38.922539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:43045 (local address 127.8.24.2:52935)
0808 16:52:38.922681 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:38.922718 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:38.923134 (+ 416us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:38.923337 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:38.923341 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:38.924031 (+ 690us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:38.924304 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:38.924310 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:38.924939 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:38.924939 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:38.925695 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:38.925697 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:38.925987 (+ 290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:38.925988 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:38.926079 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:38.926483 (+ 404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:38.926491 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:38.927669 (+ 1178us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:52:38.928974 (+ 1305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:38.928977 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:38.928979 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:38.929163 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:38.929353 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:38.929353 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:38.929353 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:38.929441 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:38.929691 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:52:38.929696 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:52:38.929822 (+ 126us) client_negotiation.cc:769] Sending connection context
0808 16:52:38.929915 (+ 93us) client_negotiation.cc:240] Negotiation successful
0808 16:52:38.930069 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":78,"threads_started":1}
I20260808 16:52:38.930823 21780 heartbeater.cc:347] Connected to a master server at 127.8.24.62:43045
I20260808 16:52:38.930914 21780 heartbeater.cc:464] Registering TS with master...
I20260808 16:52:38.931071 21780 heartbeater.cc:511] Master 127.8.24.62:43045 requested a full tablet report, sending...
I20260808 16:52:38.931392 21275 ts_manager.cc:194] Registered new tserver with Master: c4a38b53515040d4852c2933315091a6 (127.8.24.2:43333)
I20260808 16:52:38.931743 21275 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.2@KRBTEST.COM'} at 127.8.24.2:52935
WARNING: no policy specified for kudu/127.8.24.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.3.keytab.
Entry for principal kudu/127.8.24.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.3.keytab.
I20260808 16:52:38.952776 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.3:0
--local_ip_for_outbound_sockets=127.8.24.3
--webserver_interface=127.8.24.3
--webserver_port=0
--tserver_master_addrs=127.8.24.62:43045
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.3.keytab
--principal=kudu/127.8.24.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:43365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:52:39.030213 21793 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:52:39.030400 21793 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:52:39.030422 21793 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:52:39.031499 21793 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:52:39.031533 21793 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:52:39.031558 21793 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.3
I20260808 16:52:39.032879 21793 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:43365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.3.keytab
--principal=kudu/127.8.24.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.24.3
--webserver_port=0
--tserver_master_addrs=127.8.24.62:43045
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21793
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.3
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:52:39.033037 21793 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:52:39.033201 21793 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:52:39.037680 21802 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:39.037725 21803 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:39.037685 21805 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:39.037782 21793 server_base.cc:1034] running on GCE node
I20260808 16:52:39.038048 21793 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:52:39.038268 21793 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:52:39.039397 21793 hybrid_clock.cc:648] HybridClock initialized: now 1786207959039387 us; error 34 us; skew 500 ppm
Aug 08 16:52:39 dist-test-slave-6zbq krb5kdc[21210](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207959, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:52:39.041613 21793 init.cc:377] Logged in from keytab as kudu/127.8.24.3@KRBTEST.COM (short username kudu)
I20260808 16:52:39.042003 21793 webserver.cc:460] Webserver started at http://127.8.24.3:45069/ using document root <none> and password file <none>
I20260808 16:52:39.042181 21793 fs_manager.cc:359] Metadata directory not provided
I20260808 16:52:39.042224 21793 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:52:39.042378 21793 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:52:39.043316 21793 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/ts-2/data/instance:
uuid: "e40b5a6d037c4984baf0e792a0d8ed80"
format_stamp: "Formatted at 2026-08-08 16:52:39 on dist-test-slave-6zbq"
server_key: "1c35a7c586745c4cebe0186b016d5821"
server_key_iv: "2c27f5dbac85c189f6bc470ef7d09ba4"
server_key_version: "encryptionkey@0"
I20260808 16:52:39.043619 21793 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/ts-2/wal/instance:
uuid: "e40b5a6d037c4984baf0e792a0d8ed80"
format_stamp: "Formatted at 2026-08-08 16:52:39 on dist-test-slave-6zbq"
server_key: "1c35a7c586745c4cebe0186b016d5821"
server_key_iv: "2c27f5dbac85c189f6bc470ef7d09ba4"
server_key_version: "encryptionkey@0"
I20260808 16:52:39.044785 21793 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 16:52:39.045387 21815 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:39.045529 21793 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:39.045606 21793 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/ts-2/wal
uuid: "e40b5a6d037c4984baf0e792a0d8ed80"
format_stamp: "Formatted at 2026-08-08 16:52:39 on dist-test-slave-6zbq"
server_key: "1c35a7c586745c4cebe0186b016d5821"
server_key_iv: "2c27f5dbac85c189f6bc470ef7d09ba4"
server_key_version: "encryptionkey@0"
I20260808 16:52:39.045662 21793 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:52:39.051954 21793 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:52:39.053437 21793 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:52:39.053553 21793 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:52:39.053726 21793 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:52:39.053977 21793 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:52:39.054008 21793 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:39.054033 21793 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:52:39.054057 21793 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:39.059265 21793 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.3:37837
I20260808 16:52:39.059299 22003 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.3:37837 every 8 connection(s)
I20260808 16:52:39.059572 21793 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/ts-2/data/info.pb
Aug 08 16:52:39 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207959, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.3@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260808 16:52:39.066721 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 21793
I20260808 16:52:39.066829 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/ts-2/wal/instance
I20260808 16:52:39.067024 8288 external_mini_cluster.cc:1428] Setting key 361f8defac5e7666c1ca32412b47720b
I20260808 16:52:39.067871 21354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:39.060158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:51367 (local address 127.8.24.62:43045)
0808 16:52:39.060210 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:39.060212 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:39.060379 (+ 167us) server_negotiation.cc:407] Connection header received
0808 16:52:39.060865 (+ 486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:39.060865 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:39.060875 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:39.060908 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:39.061695 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:39.062224 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:39.063197 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:39.063378 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:39.063424 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:39.065209 (+ 1785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:39.065210 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:39.065211 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:39.065217 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:39.066463 (+ 1246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:39.066877 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:39.066878 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:39.066878 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:39.066936 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:39.067263 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:39.067264 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:39.067264 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:39.067397 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:39.067457 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:39.067708 (+ 251us) server_negotiation.cc:299] Negotiation successful
0808 16:52:39.067780 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 16:52:39.068053 22009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:39.060135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:43045 (local address 127.8.24.3:51367)
0808 16:52:39.060272 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:39.060308 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:39.060768 (+ 460us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:39.060961 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:39.060964 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:39.061467 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:39.061633 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:39.061637 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:39.062287 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:39.062288 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:39.063135 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:39.063137 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:39.063431 (+ 294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:39.063432 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:39.063531 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:39.063879 (+ 348us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:39.063887 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:39.065132 (+ 1245us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:52:39.066520 (+ 1388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:39.066525 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:39.066527 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:39.066791 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:39.066982 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:39.066983 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:39.066983 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:39.067106 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:39.067456 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:52:39.067462 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:52:39.067585 (+ 123us) client_negotiation.cc:769] Sending connection context
0808 16:52:39.067687 (+ 102us) client_negotiation.cc:240] Negotiation successful
0808 16:52:39.067849 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":65,"threads_started":1}
I20260808 16:52:39.068516 22005 heartbeater.cc:347] Connected to a master server at 127.8.24.62:43045
I20260808 16:52:39.068589 22005 heartbeater.cc:464] Registering TS with master...
I20260808 16:52:39.068713 22005 heartbeater.cc:511] Master 127.8.24.62:43045 requested a full tablet report, sending...
I20260808 16:52:39.068975 21275 ts_manager.cc:194] Registered new tserver with Master: e40b5a6d037c4984baf0e792a0d8ed80 (127.8.24.3:37837)
I20260808 16:52:39.069305 21275 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.3@KRBTEST.COM'} at 127.8.24.3:51367
I20260808 16:52:39.078513 8288 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.8.24.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.24.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.24.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/krb5kdc/oryx_127.8.24.1.keytab.
Entry for principal oryx/127.8.24.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/krb5kdc/oryx_127.8.24.1.keytab.
WARNING: no policy specified for oryx/127.8.24.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.24.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.24.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/krb5kdc/oryx_127.8.24.2.keytab.
Entry for principal oryx/127.8.24.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/krb5kdc/oryx_127.8.24.2.keytab.
WARNING: no policy specified for oryx/127.8.24.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.24.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.24.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/krb5kdc/oryx_127.8.24.3.keytab.
Entry for principal oryx/127.8.24.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/krb5kdc/oryx_127.8.24.3.keytab.
I20260808 16:52:39.140586 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 21376
I20260808 16:52:39.154953 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 21581
I20260808 16:52:39.168583 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 21793
I20260808 16:52:39.174081 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 21227
I20260808 16:52:39.183451 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:43045
--webserver_interface=127.8.24.62
--webserver_port=42887
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:43365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:43045
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:52:39.264073 22021 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:52:39.264261 22021 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:52:39.264281 22021 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:52:39.265378 22021 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 16:52:39.265417 22021 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:52:39.265430 22021 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:52:39.265442 22021 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 16:52:39.265465 22021 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 16:52:39.266956 22021 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:43365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:43045
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:43045
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.24.62
--webserver_port=42887
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22021
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:52:39.267174 22021 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:52:39.267371 22021 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:52:39.272037 22031 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:39.272066 22030 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:39.272166 22035 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:39.272433 22021 server_base.cc:1034] running on GCE node
I20260808 16:52:39.272671 22021 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:52:39.272940 22021 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:52:39.274085 22021 hybrid_clock.cc:648] HybridClock initialized: now 1786207959274068 us; error 27 us; skew 500 ppm
Aug 08 16:52:39 dist-test-slave-6zbq krb5kdc[21210](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207959, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:52:39.276432 22021 init.cc:377] Logged in from keytab as kudu/127.8.24.62@KRBTEST.COM (short username kudu)
I20260808 16:52:39.276839 22021 webserver.cc:460] Webserver started at http://127.8.24.62:42887/ using document root <none> and password file <none>
I20260808 16:52:39.277050 22021 fs_manager.cc:359] Metadata directory not provided
I20260808 16:52:39.277113 22021 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:52:39.278327 22021 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 16:52:39.278944 22046 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:39.279081 22021 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:39.279152 22021 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/master-0/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/master-0/wal
uuid: "f6eacb891fa04c52b99587ac9f7c0587"
format_stamp: "Formatted at 2026-08-08 16:52:38 on dist-test-slave-6zbq"
server_key: "caa3eac9b2669beb2db8e221d9db264c"
server_key_iv: "ecce497d48d834b54482c958cafbc79c"
server_key_version: "encryptionkey@0"
I20260808 16:52:39.279459 22021 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:52:39.323577 22021 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:52:39.325620 22021 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:52:39.325754 22021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:52:39.329533 22021 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.62:43045
I20260808 16:52:39.329566 22130 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.62:43045 every 8 connection(s)
I20260808 16:52:39.329918 22021 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
I20260808 16:52:39.331322 22132 sys_catalog.cc:263] Verifying existing consensus state
I20260808 16:52:39.331977 22132 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6eacb891fa04c52b99587ac9f7c0587: Bootstrap starting.
I20260808 16:52:39.338258 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 22021
I20260808 16:52:39.338617 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.1:43545
--local_ip_for_outbound_sockets=127.8.24.1
--tserver_master_addrs=127.8.24.62:43045
--webserver_port=46041
--webserver_interface=127.8.24.1
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.1.keytab
--principal=kudu/127.8.24.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:43365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.8.24.1
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/krb5kdc/oryx_127.8.24.1.keytab with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 16:52:39.342226 22132 log.cc:824] T 00000000000000000000000000000000 P f6eacb891fa04c52b99587ac9f7c0587: Log is configured to *not* fsync() on all Append() calls
I20260808 16:52:39.343816 22132 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6eacb891fa04c52b99587ac9f7c0587: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 16:52:39.344058 22132 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6eacb891fa04c52b99587ac9f7c0587: Bootstrap complete.
I20260808 16:52:39.346110 22132 raft_consensus.cc:348] T 00000000000000000000000000000000 P f6eacb891fa04c52b99587ac9f7c0587 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6eacb891fa04c52b99587ac9f7c0587" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 43045 } }
I20260808 16:52:39.346333 22132 raft_consensus.cc:729] T 00000000000000000000000000000000 P f6eacb891fa04c52b99587ac9f7c0587 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6eacb891fa04c52b99587ac9f7c0587, State: Initialized, Role: FOLLOWER
I20260808 16:52:39.346508 22132 consensus_queue.cc:260] T 00000000000000000000000000000000 P f6eacb891fa04c52b99587ac9f7c0587 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6eacb891fa04c52b99587ac9f7c0587" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 43045 } }
I20260808 16:52:39.346568 22132 raft_consensus.cc:388] T 00000000000000000000000000000000 P f6eacb891fa04c52b99587ac9f7c0587 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 16:52:39.346614 22132 raft_consensus.cc:482] T 00000000000000000000000000000000 P f6eacb891fa04c52b99587ac9f7c0587 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 16:52:39.346671 22132 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f6eacb891fa04c52b99587ac9f7c0587 [term 1 FOLLOWER]: Advancing to term 2
I20260808 16:52:39.347633 22132 raft_consensus.cc:504] T 00000000000000000000000000000000 P f6eacb891fa04c52b99587ac9f7c0587 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6eacb891fa04c52b99587ac9f7c0587" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 43045 } }
I20260808 16:52:39.347730 22132 leader_election.cc:302] T 00000000000000000000000000000000 P f6eacb891fa04c52b99587ac9f7c0587 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f6eacb891fa04c52b99587ac9f7c0587; no voters:
I20260808 16:52:39.347894 22132 leader_election.cc:288] T 00000000000000000000000000000000 P f6eacb891fa04c52b99587ac9f7c0587 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 16:52:39.347942 22137 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f6eacb891fa04c52b99587ac9f7c0587 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 16:52:39.348067 22137 raft_consensus.cc:686] T 00000000000000000000000000000000 P f6eacb891fa04c52b99587ac9f7c0587 [term 2 LEADER]: Becoming Leader. State: Replica: f6eacb891fa04c52b99587ac9f7c0587, State: Running, Role: LEADER
I20260808 16:52:39.348160 22137 consensus_queue.cc:237] T 00000000000000000000000000000000 P f6eacb891fa04c52b99587ac9f7c0587 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6eacb891fa04c52b99587ac9f7c0587" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 43045 } }
I20260808 16:52:39.348160 22132 sys_catalog.cc:564] T 00000000000000000000000000000000 P f6eacb891fa04c52b99587ac9f7c0587 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 16:52:39.348479 22137 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6eacb891fa04c52b99587ac9f7c0587 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f6eacb891fa04c52b99587ac9f7c0587. Latest consensus state: current_term: 2 leader_uuid: "f6eacb891fa04c52b99587ac9f7c0587" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6eacb891fa04c52b99587ac9f7c0587" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 43045 } } }
I20260808 16:52:39.348546 22137 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6eacb891fa04c52b99587ac9f7c0587 [sys.catalog]: This master's current role is: LEADER
I20260808 16:52:39.348732 22138 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6eacb891fa04c52b99587ac9f7c0587 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "f6eacb891fa04c52b99587ac9f7c0587" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6eacb891fa04c52b99587ac9f7c0587" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 43045 } } }
I20260808 16:52:39.348875 22138 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6eacb891fa04c52b99587ac9f7c0587 [sys.catalog]: This master's current role is: LEADER
I20260808 16:52:39.349011 22147 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 16:52:39.349740 22147 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 16:52:39.349856 22147 catalog_manager.cc:1259] Loaded cluster ID: 65134025533c43eeb8d8a171556cbbae
I20260808 16:52:39.349888 22147 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 16:52:39.351068 22147 catalog_manager.cc:1504] Loading token signing keys...
I20260808 16:52:39.351248 22147 catalog_manager.cc:5879] T 00000000000000000000000000000000 P f6eacb891fa04c52b99587ac9f7c0587: Loaded TSK: 0
I20260808 16:52:39.351446 22147 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 16:52:39.437129 22134 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:52:39.437325 22134 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:52:39.437356 22134 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:52:39.438460 22134 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:52:39.438496 22134 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:52:39.438531 22134 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.1
I20260808 16:52:39.439893 22134 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:43365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/krb5kdc/oryx_127.8.24.1.keytab
--principal=oryx/127.8.24.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.1:43545
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.24.1
--webserver_port=46041
--tserver_master_addrs=127.8.24.62:43045
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22134
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.1
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:52:39.440096 22134 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:52:39.440290 22134 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:52:39.444862 22179 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:39.444854 22175 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:39.444865 22176 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:39.445152 22134 server_base.cc:1034] running on GCE node
I20260808 16:52:39.445338 22134 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:52:39.445552 22134 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:52:39.446691 22134 hybrid_clock.cc:648] HybridClock initialized: now 1786207959446677 us; error 32 us; skew 500 ppm
Aug 08 16:52:39 dist-test-slave-6zbq krb5kdc[21210](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207959, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.24.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:52:39.449160 22134 init.cc:377] Logged in from keytab as oryx/127.8.24.1@KRBTEST.COM (short username oryx)
I20260808 16:52:39.449615 22134 webserver.cc:460] Webserver started at http://127.8.24.1:46041/ using document root <none> and password file <none>
I20260808 16:52:39.449826 22134 fs_manager.cc:359] Metadata directory not provided
I20260808 16:52:39.449879 22134 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:52:39.451156 22134 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 16:52:39.451737 22191 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:39.451915 22134 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 16:52:39.451995 22134 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/ts-0/wal
uuid: "4e8635a62a7c43b4b314aedf778f9eb0"
format_stamp: "Formatted at 2026-08-08 16:52:38 on dist-test-slave-6zbq"
server_key: "45adf9628f2466ab5a4da8cc429998b9"
server_key_iv: "a3ef720e0fa7aecebea922eb2c6bca14"
server_key_version: "encryptionkey@0"
I20260808 16:52:39.452280 22134 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:52:39.475730 22134 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:52:39.477573 22134 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:52:39.477698 22134 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:52:39.477900 22134 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:52:39.478165 22134 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:52:39.478199 22134 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:39.478224 22134 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:52:39.478248 22134 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:39.484113 22134 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.1:43545
I20260808 16:52:39.484211 22389 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.1:43545 every 8 connection(s)
I20260808 16:52:39.484499 22134 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/ts-0/data/info.pb
Aug 08 16:52:39 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.1@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:52:39 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.1@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 16:52:39.492515 22396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:39.485200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:43045 (local address 127.8.24.1:41507)
0808 16:52:39.485368 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:39.485407 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:39.485843 (+ 436us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:39.486228 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:39.486231 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:39.486796 (+ 565us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:39.486973 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:39.486978 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:39.488502 (+ 1524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:39.488503 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:39.489233 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:39.489236 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:39.489928 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:39.489928 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:39.490018 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:39.490463 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:39.490471 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:39.492385 (+ 1914us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.24.62:43045: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":74,"threads_started":1}
W20260808 16:52:39.492744 22392 heartbeater.cc:644] Failed to heartbeat to 127.8.24.62:43045 (0 consecutive failures): Not authorized: Failed to ping master at 127.8.24.62:43045: Client connection negotiation failed: client connection to 127.8.24.62:43045: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
W20260808 16:52:39.492901 22397 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:52:39.485264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:41507 (local address 127.8.24.62:43045)
0808 16:52:39.485440 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:39.485448 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:39.485483 (+ 35us) server_negotiation.cc:407] Connection header received
0808 16:52:39.486007 (+ 524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:39.486013 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:39.486063 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:39.486200 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:39.487075 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:39.488414 (+ 1339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:39.489309 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:39.489857 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:39.489959 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:39.492496 (+ 2537us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.24.1:41507: BlockingRecv error: recv got EOF from 127.8.24.1:41507 (error 108)
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":81,"threads_started":1}
I20260808 16:52:39.494555 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 22134
I20260808 16:52:39.494834 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.2:43333
--local_ip_for_outbound_sockets=127.8.24.2
--tserver_master_addrs=127.8.24.62:43045
--webserver_port=37195
--webserver_interface=127.8.24.2
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.2.keytab
--principal=kudu/127.8.24.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:43365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.8.24.2
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/krb5kdc/oryx_127.8.24.2.keytab with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 16:52:39 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.1@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:52:39 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.1@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 16:52:39.496876 22396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:39.492986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:43045 (local address 127.8.24.1:56257)
0808 16:52:39.493036 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:39.493047 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:39.493088 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:39.493208 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:39.493209 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:39.493315 (+ 106us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:39.493357 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:39.493358 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:39.493900 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:39.493901 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:39.494461 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:39.494464 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:39.494795 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:39.494796 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:39.494864 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:39.495288 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:39.495293 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:39.496818 (+ 1525us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.24.62:43045: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":35}
W20260808 16:52:39.496954 22397 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:52:39.493057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:56257 (local address 127.8.24.62:43045)
0808 16:52:39.493109 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:39.493111 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:39.493121 (+ 10us) server_negotiation.cc:407] Connection header received
0808 16:52:39.493141 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:39.493141 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:39.493151 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:39.493195 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:39.493411 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:39.493850 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:39.494529 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:39.494735 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:39.494780 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:39.496910 (+ 2130us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.24.1:56257: BlockingRecv error: recv got EOF from 127.8.24.1:56257 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Aug 08 16:52:39 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.1@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:52:39 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.1@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 16:52:39.501375 22399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:39.497079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:43045 (local address 127.8.24.1:46493)
0808 16:52:39.497213 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:39.497232 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:39.497296 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:39.497404 (+ 108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:39.497404 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:39.497561 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:39.497618 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:39.497620 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:39.498173 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:39.498174 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:39.498995 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:39.498998 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:39.499319 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:39.499320 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:39.499378 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:39.499710 (+ 332us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:39.499714 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:39.501301 (+ 1587us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.24.62:43045: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":79,"threads_started":1}
W20260808 16:52:39.501405 22397 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:52:39.497212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:46493 (local address 127.8.24.62:43045)
0808 16:52:39.497255 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:39.497257 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:39.497282 (+ 25us) server_negotiation.cc:407] Connection header received
0808 16:52:39.497347 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:39.497348 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:39.497355 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:39.497396 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:39.497681 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:39.498101 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:39.499068 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:39.499237 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:39.499284 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:39.501369 (+ 2085us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.24.1:46493: BlockingRecv error: recv got EOF from 127.8.24.1:46493 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
W20260808 16:52:39.501573 22392 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 16:52:39.577013 22398 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:52:39.577207 22398 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:52:39.577237 22398 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:52:39.578378 22398 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:52:39.578416 22398 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:52:39.578450 22398 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.2
I20260808 16:52:39.579857 22398 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:43365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/krb5kdc/oryx_127.8.24.2.keytab
--principal=oryx/127.8.24.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.2:43333
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.24.2
--webserver_port=37195
--tserver_master_addrs=127.8.24.62:43045
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22398
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.2
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:52:39.580080 22398 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:52:39.580295 22398 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:52:39.584976 22407 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:39.585120 22398 server_base.cc:1034] running on GCE node
W20260808 16:52:39.585041 22409 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:39.584985 22406 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:39.585399 22398 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:52:39.585600 22398 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:52:39.586726 22398 hybrid_clock.cc:648] HybridClock initialized: now 1786207959586714 us; error 27 us; skew 500 ppm
Aug 08 16:52:39 dist-test-slave-6zbq krb5kdc[21210](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207959, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.24.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:52:39.588892 22398 init.cc:377] Logged in from keytab as oryx/127.8.24.2@KRBTEST.COM (short username oryx)
I20260808 16:52:39.589262 22398 webserver.cc:460] Webserver started at http://127.8.24.2:37195/ using document root <none> and password file <none>
I20260808 16:52:39.589470 22398 fs_manager.cc:359] Metadata directory not provided
I20260808 16:52:39.589526 22398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:52:39.590817 22398 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 16:52:39.591388 22420 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:39.591540 22398 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:39.591617 22398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/ts-1/wal
uuid: "c4a38b53515040d4852c2933315091a6"
format_stamp: "Formatted at 2026-08-08 16:52:38 on dist-test-slave-6zbq"
server_key: "c19c62f5bdb506be148de2a5dd391227"
server_key_iv: "7d915a357c612752c27ed4c4089e277b"
server_key_version: "encryptionkey@0"
I20260808 16:52:39.591893 22398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:52:39.611590 22398 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:52:39.613179 22398 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:52:39.613281 22398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:52:39.613442 22398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:52:39.613668 22398 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:52:39.613695 22398 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:39.613715 22398 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:52:39.613730 22398 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:39.618834 22398 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.2:43333
I20260808 16:52:39.618870 22602 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.2:43333 every 8 connection(s)
I20260808 16:52:39.619174 22398 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/ts-1/data/info.pb
I20260808 16:52:39.620652 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 22398
I20260808 16:52:39.620896 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.3:37837
--local_ip_for_outbound_sockets=127.8.24.3
--tserver_master_addrs=127.8.24.62:43045
--webserver_port=45069
--webserver_interface=127.8.24.3
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.3.keytab
--principal=kudu/127.8.24.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:43365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.8.24.3
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/krb5kdc/oryx_127.8.24.3.keytab with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 16:52:39 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.2@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:52:39 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.2@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 16:52:39.626230 22609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:39.619761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:43045 (local address 127.8.24.2:39201)
0808 16:52:39.619926 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:39.619966 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:39.620418 (+ 452us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:39.620675 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:39.620679 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:39.621280 (+ 601us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:39.621454 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:39.621458 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:39.622094 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:39.622095 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:39.623276 (+ 1181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:39.623279 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:39.623633 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:39.623634 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:39.623734 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:39.624174 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:39.624183 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:39.626102 (+ 1919us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.24.62:43045: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":81,"threads_started":1}
W20260808 16:52:39.626286 22397 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:52:39.619793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:39201 (local address 127.8.24.62:43045)
0808 16:52:39.619845 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:39.619847 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:39.620033 (+ 186us) server_negotiation.cc:407] Connection header received
0808 16:52:39.620528 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:39.620529 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:39.620546 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:39.620596 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:39.621539 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:39.622016 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:39.623361 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:39.623556 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:39.623616 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:39.626222 (+ 2606us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.24.2:39201: BlockingRecv error: recv got EOF from 127.8.24.2:39201 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
W20260808 16:52:39.626479 22604 heartbeater.cc:644] Failed to heartbeat to 127.8.24.62:43045 (0 consecutive failures): Not authorized: Failed to ping master at 127.8.24.62:43045: Client connection negotiation failed: client connection to 127.8.24.62:43045: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Aug 08 16:52:39 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.2@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:52:39 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.2@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 16:52:39.630698 22609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:39.626709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:43045 (local address 127.8.24.2:49937)
0808 16:52:39.626816 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:39.626824 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:39.626858 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:39.626973 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:39.626974 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:39.627072 (+ 98us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:39.627109 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:39.627110 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:39.627908 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:39.627909 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:39.628427 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:39.628429 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:39.628841 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:39.628841 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:39.628881 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:39.629201 (+ 320us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:39.629205 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:39.630659 (+ 1454us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.24.62:43045: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":90}
W20260808 16:52:39.630811 22397 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:52:39.626775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:49937 (local address 127.8.24.62:43045)
0808 16:52:39.626834 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:39.626837 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:39.626849 (+ 12us) server_negotiation.cc:407] Connection header received
0808 16:52:39.626902 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:39.626903 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:39.626913 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:39.626972 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:39.627171 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:39.627837 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:39.628499 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:39.628775 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:39.628846 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:39.630757 (+ 1911us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.24.2:49937: BlockingRecv error: recv got EOF from 127.8.24.2:49937 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Aug 08 16:52:39 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.2@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:52:39 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.2@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 16:52:39.635200 22613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:39.630888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:43045 (local address 127.8.24.2:51579)
0808 16:52:39.631023 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:39.631033 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:39.631083 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:39.631221 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:39.631222 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:39.631387 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:39.631439 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:39.631441 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:39.631978 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:39.631979 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:39.632850 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:39.632852 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:39.633148 (+ 296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:39.633149 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:39.633205 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:39.633528 (+ 323us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:39.633532 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:39.635117 (+ 1585us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.24.62:43045: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":64,"threads_started":1}
W20260808 16:52:39.635223 22397 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:52:39.631034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:51579 (local address 127.8.24.62:43045)
0808 16:52:39.631069 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:39.631071 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:39.631080 (+ 9us) server_negotiation.cc:407] Connection header received
0808 16:52:39.631144 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:39.631145 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:39.631153 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:39.631203 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:39.631505 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:39.631919 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:39.632916 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:39.633077 (+ 161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:39.633113 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:39.635186 (+ 2073us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.24.2:51579: BlockingRecv error: recv got EOF from 127.8.24.2:51579 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260808 16:52:39.635428 22604 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 16:52:39.702431 22612 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:52:39.702613 22612 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:52:39.702649 22612 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:52:39.703760 22612 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:52:39.703799 22612 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:52:39.703832 22612 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.3
I20260808 16:52:39.705185 22612 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:43365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/krb5kdc/oryx_127.8.24.3.keytab
--principal=oryx/127.8.24.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.3:37837
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.24.3
--webserver_port=45069
--tserver_master_addrs=127.8.24.62:43045
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22612
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.3
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:52:39.705400 22612 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:52:39.705595 22612 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:52:39.710170 22627 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:39.710208 22612 server_base.cc:1034] running on GCE node
W20260808 16:52:39.710179 22625 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:39.710181 22624 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:39.710570 22612 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:52:39.710788 22612 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:52:39.711944 22612 hybrid_clock.cc:648] HybridClock initialized: now 1786207959711924 us; error 27 us; skew 500 ppm
Aug 08 16:52:39 dist-test-slave-6zbq krb5kdc[21210](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207959, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.24.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:52:39.714213 22612 init.cc:377] Logged in from keytab as oryx/127.8.24.3@KRBTEST.COM (short username oryx)
I20260808 16:52:39.714635 22612 webserver.cc:460] Webserver started at http://127.8.24.3:45069/ using document root <none> and password file <none>
I20260808 16:52:39.714836 22612 fs_manager.cc:359] Metadata directory not provided
I20260808 16:52:39.714900 22612 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:52:39.716158 22612 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 16:52:39.716681 22640 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:39.716837 22612 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 16:52:39.716917 22612 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/ts-2/wal
uuid: "e40b5a6d037c4984baf0e792a0d8ed80"
format_stamp: "Formatted at 2026-08-08 16:52:39 on dist-test-slave-6zbq"
server_key: "1c35a7c586745c4cebe0186b016d5821"
server_key_iv: "2c27f5dbac85c189f6bc470ef7d09ba4"
server_key_version: "encryptionkey@0"
I20260808 16:52:39.717227 22612 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:52:39.733824 22612 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:52:39.735836 22612 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:52:39.735963 22612 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:52:39.736159 22612 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:52:39.736424 22612 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:52:39.736456 22612 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:39.736485 22612 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:52:39.736506 22612 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:39.741758 22612 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.3:37837
I20260808 16:52:39.741818 22817 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.3:37837 every 8 connection(s)
I20260808 16:52:39.742113 22612 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786207919384338-8288-0/minicluster-data/ts-2/data/info.pb
I20260808 16:52:39.746333 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 22612
Aug 08 16:52:39 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.3@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:52:39 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.3@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
W20260808 16:52:39.748963 22397 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:52:39.742783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:41739 (local address 127.8.24.62:43045)
0808 16:52:39.742839 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:39.742842 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:39.743018 (+ 176us) server_negotiation.cc:407] Connection header received
0808 16:52:39.743472 (+ 454us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:39.743472 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:39.743480 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:39.743529 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:39.744358 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:39.744826 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:39.745748 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:39.745924 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:39.745963 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:39.748883 (+ 2920us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.24.3:41739: BlockingRecv error: recv got EOF from 127.8.24.3:41739 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 16:52:39.748994 22821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:39.742758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:43045 (local address 127.8.24.3:41739)
0808 16:52:39.742915 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:39.742953 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:39.743398 (+ 445us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:39.743580 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:39.743583 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:39.744103 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:39.744294 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:39.744298 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:39.744906 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:39.744906 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:39.745689 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:39.745691 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:39.745974 (+ 283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:39.745974 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:39.746062 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:39.746530 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:39.746538 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:39.748780 (+ 2242us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.24.62:43045: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":70,"threads_started":1}
W20260808 16:52:39.749200 22818 heartbeater.cc:644] Failed to heartbeat to 127.8.24.62:43045 (0 consecutive failures): Not authorized: Failed to ping master at 127.8.24.62:43045: Client connection negotiation failed: client connection to 127.8.24.62:43045: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Aug 08 16:52:39 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.3@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:52:39 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.3@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 16:52:39.753077 22821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:39.749400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:43045 (local address 127.8.24.3:55409)
0808 16:52:39.749438 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:39.749447 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:39.749479 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:39.749622 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:39.749623 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:39.749733 (+ 110us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:39.749776 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:39.749777 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:39.750357 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:39.750358 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:39.750839 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:39.750841 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:39.751163 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:39.751163 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:39.751230 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:39.751524 (+ 294us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:39.751527 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:39.753018 (+ 1491us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.24.62:43045: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":26}
W20260808 16:52:39.753149 22397 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:52:39.749442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:55409 (local address 127.8.24.62:43045)
0808 16:52:39.749474 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:39.749475 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:39.749482 (+ 7us) server_negotiation.cc:407] Connection header received
0808 16:52:39.749533 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:39.749533 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:39.749542 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:39.749583 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:39.749845 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:39.750274 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:39.750909 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:39.751099 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:39.751150 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:39.753099 (+ 1949us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.24.3:55409: BlockingRecv error: recv got EOF from 127.8.24.3:55409 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 16:52:39.753981 22823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:39.746842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32876 (local address 127.8.24.62:43045)
0808 16:52:39.746968 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:39.746970 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:39.746978 (+ 8us) server_negotiation.cc:407] Connection header received
0808 16:52:39.747001 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:39.747002 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:39.747013 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:39.747087 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:39.747784 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:39.748374 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:39.749293 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:39.749578 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:39.749632 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:39.750757 (+ 1125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:39.750761 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:39.750761 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:39.750772 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:39.752421 (+ 1649us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:39.752721 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:39.752724 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:39.752725 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:39.752842 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:39.753016 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:39.753017 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:39.753017 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:39.753280 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:39.753410 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:39.753836 (+ 426us) server_negotiation.cc:299] Negotiation successful
0808 16:52:39.753923 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":67,"threads_started":1}
Aug 08 16:52:39 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.3@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:52:39 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.3@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 16:52:39.757028 22821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:39.753366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:43045 (local address 127.8.24.3:40403)
0808 16:52:39.753402 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:39.753413 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:39.753447 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:39.753545 (+ 98us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:39.753545 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:39.753672 (+ 127us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:39.753701 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:39.753702 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:39.754395 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:39.754396 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:39.755077 (+ 681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:39.755079 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:39.755362 (+ 283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:39.755362 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:39.755395 (+ 33us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:39.755703 (+ 308us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:39.755707 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:39.756989 (+ 1282us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.24.62:43045: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":19}
W20260808 16:52:39.757145 22397 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:52:39.753412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:40403 (local address 127.8.24.62:43045)
0808 16:52:39.753445 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:39.753447 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:39.753455 (+ 8us) server_negotiation.cc:407] Connection header received
0808 16:52:39.753495 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:39.753495 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:39.753502 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:39.753540 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:39.753767 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:39.754302 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:39.755147 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:39.755315 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:39.755358 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:39.757089 (+ 1731us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.24.3:40403: BlockingRecv error: recv got EOF from 127.8.24.3:40403 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260808 16:52:39.757184 22818 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 08 16:52:40 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.1@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:52:40 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.1@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 16:52:40.505925 22824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:40.501883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:43045 (local address 127.8.24.1:55887)
0808 16:52:40.501994 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:40.502004 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:40.502049 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:40.502204 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:40.502205 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:40.502382 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:40.502433 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:40.502434 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:40.503035 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:40.503036 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:40.503563 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:40.503564 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:40.503840 (+ 276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:40.503841 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:40.503888 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:40.504307 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:40.504312 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:40.505860 (+ 1548us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.24.62:43045: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":58,"threads_started":1}
W20260808 16:52:40.506073 22825 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:52:40.501953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:55887 (local address 127.8.24.62:43045)
0808 16:52:40.502051 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:40.502052 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:40.502077 (+ 25us) server_negotiation.cc:407] Connection header received
0808 16:52:40.502097 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:40.502097 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:40.502109 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:40.502156 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:40.502492 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:40.502974 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:40.503614 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:40.503795 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:40.503836 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:40.505945 (+ 2109us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.24.1:55887: BlockingRecv error: recv got EOF from 127.8.24.1:55887 (error 108)
Metrics: {"server-negotiator.queue_time_us":80,"thread_start_us":48,"threads_started":1}
Aug 08 16:52:40 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.2@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:52:40 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.2@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 16:52:40.639575 22826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:40.635715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:43045 (local address 127.8.24.2:45077)
0808 16:52:40.635829 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:40.635839 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:40.635888 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:40.635994 (+ 106us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:40.635995 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:40.636139 (+ 144us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:40.636184 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:40.636186 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:40.636762 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:40.636763 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:40.637271 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:40.637272 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:40.637541 (+ 269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:40.637541 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:40.637590 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:40.637960 (+ 370us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:40.637964 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:40.639525 (+ 1561us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.24.62:43045: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":54,"threads_started":1}
W20260808 16:52:40.639669 22825 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:52:40.635766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:45077 (local address 127.8.24.62:43045)
0808 16:52:40.635809 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:40.635811 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:40.635875 (+ 64us) server_negotiation.cc:407] Connection header received
0808 16:52:40.635925 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:40.635926 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:40.635936 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:40.635984 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:40.636230 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:40.636709 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:40.637327 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:40.637496 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:40.637538 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:40.639611 (+ 2073us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.24.2:45077: BlockingRecv error: recv got EOF from 127.8.24.2:45077 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
Aug 08 16:52:40 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.3@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:52:40 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.3@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 16:52:40.762897 22827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:40.757526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:43045 (local address 127.8.24.3:54845)
0808 16:52:40.757662 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:40.757670 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:40.757735 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:40.757885 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:40.757886 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:40.758074 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:40.758132 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:40.758134 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:40.758893 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:40.758893 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:40.759564 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:40.759566 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:40.759921 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:40.759922 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:40.759969 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:40.760507 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:40.760513 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:40.762819 (+ 2306us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.24.62:43045: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":73,"threads_started":1}
W20260808 16:52:40.762988 22825 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:52:40.757597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:54845 (local address 127.8.24.62:43045)
0808 16:52:40.757658 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:40.757659 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:40.757727 (+ 68us) server_negotiation.cc:407] Connection header received
0808 16:52:40.757794 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:40.757794 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:40.757812 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:40.757875 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:40.758200 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:40.758822 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:40.759641 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:40.759855 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:40.759916 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:40.762914 (+ 2998us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.24.3:54845: BlockingRecv error: recv got EOF from 127.8.24.3:54845 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
Aug 08 16:52:41 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.1@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:52:41 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.1@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 16:52:41.513079 22828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:41.506422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:43045 (local address 127.8.24.1:53307)
0808 16:52:41.506630 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:41.506648 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:41.506722 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:41.506952 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:41.506953 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:41.507211 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:41.507292 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:41.507294 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:41.508241 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:41.508242 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:41.509074 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:41.509076 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:41.509645 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:41.509646 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:41.509701 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:41.510466 (+ 765us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:41.510470 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:41.513000 (+ 2530us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.24.62:43045: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":135,"threads_started":1}
W20260808 16:52:41.513247 22830 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:52:41.506514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:53307 (local address 127.8.24.62:43045)
0808 16:52:41.506681 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:41.506683 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:41.506699 (+ 16us) server_negotiation.cc:407] Connection header received
0808 16:52:41.506841 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:41.506841 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:41.506869 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:41.506944 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:41.507384 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:41.508144 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:41.509194 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:41.509558 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:41.509635 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:41.513125 (+ 3490us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.24.1:53307: BlockingRecv error: recv got EOF from 127.8.24.1:53307 (error 108)
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":91,"threads_started":1}
Aug 08 16:52:41 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.2@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:52:41 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.2@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 16:52:41.647025 22835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:41.640140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:43045 (local address 127.8.24.2:39627)
0808 16:52:41.640369 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:41.640378 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:41.640457 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:41.640772 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:41.640773 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:41.641024 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:41.641114 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:41.641116 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:41.641967 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:41.641967 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:41.642838 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:41.642850 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:41.643356 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:41.643356 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:41.643412 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:41.644131 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:41.644137 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:41.646879 (+ 2742us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.24.62:43045: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":131,"threads_started":1}
W20260808 16:52:41.647181 22830 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:52:41.640256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:39627 (local address 127.8.24.62:43045)
0808 16:52:41.640363 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:41.640366 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:41.640449 (+ 83us) server_negotiation.cc:407] Connection header received
0808 16:52:41.640609 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:41.640610 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:41.640634 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:41.640747 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:41.641247 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:41.641858 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:41.642958 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:41.643272 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:41.643348 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:41.647054 (+ 3706us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.24.2:39627: BlockingRecv error: recv got EOF from 127.8.24.2:39627 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 08 16:52:41 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.3@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:52:41 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.3@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 16:52:41.771292 22836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:41.763495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:43045 (local address 127.8.24.3:40971)
0808 16:52:41.763703 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:41.763723 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:41.763823 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:41.764041 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:41.764042 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:41.764332 (+ 290us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:41.764417 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:41.764420 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:41.765717 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:41.765719 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:41.767006 (+ 1287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:41.767010 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:41.767737 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:41.767738 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:41.767835 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:41.768501 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:41.768506 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:41.771213 (+ 2707us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.24.62:43045: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":81,"threads_started":1}
W20260808 16:52:41.771518 22830 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:52:41.763665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:40971 (local address 127.8.24.62:43045)
0808 16:52:41.763759 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:41.763761 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:41.763779 (+ 18us) server_negotiation.cc:407] Connection header received
0808 16:52:41.763907 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:41.763909 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:41.763934 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:41.764036 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:41.764515 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:41.765557 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:41.767172 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:41.767601 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:41.767726 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:41.771417 (+ 3691us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.24.3:40971: BlockingRecv error: recv got EOF from 127.8.24.3:40971 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 08 16:52:42 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.1@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:52:42 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.1@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 16:52:42.518271 22838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:42.513561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:43045 (local address 127.8.24.1:44267)
0808 16:52:42.513731 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:42.513760 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:42.513846 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:42.513978 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:42.513979 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:42.514141 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:42.514189 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:42.514191 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:42.514851 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:42.514851 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:42.515368 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:42.515369 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:42.515766 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:42.515767 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:42.515808 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:42.516273 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:42.516278 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:42.518193 (+ 1915us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.24.62:43045: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":76,"threads_started":1}
W20260808 16:52:42.518558 22839 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:52:42.513632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:44267 (local address 127.8.24.62:43045)
0808 16:52:42.513837 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:42.513839 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:42.513848 (+ 9us) server_negotiation.cc:407] Connection header received
0808 16:52:42.513891 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:42.513891 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:42.513903 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:42.513965 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:42.514259 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:42.514797 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:42.515472 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:42.515692 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:42.515753 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:42.518495 (+ 2742us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.24.1:44267: BlockingRecv error: recv got EOF from 127.8.24.1:44267 (error 108)
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":94,"threads_started":1}
Aug 08 16:52:42 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.2@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:52:42 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.2@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 16:52:42.657457 22841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:42.647714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:43045 (local address 127.8.24.2:48713)
0808 16:52:42.647901 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:42.647915 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:42.648019 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:42.648332 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:42.648333 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:42.648838 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:42.649048 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:42.649051 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:42.650898 (+ 1847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:42.650907 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:42.652424 (+ 1517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:42.652434 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:42.653506 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:42.653508 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:42.653730 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:42.654815 (+ 1085us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:42.654821 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:42.657384 (+ 2563us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.24.62:43045: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":105,"threads_started":1}
W20260808 16:52:42.657598 22839 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:52:42.647820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:48713 (local address 127.8.24.62:43045)
0808 16:52:42.647906 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:42.647908 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:42.647989 (+ 81us) server_negotiation.cc:407] Connection header received
0808 16:52:42.648105 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:42.648106 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:42.648194 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:42.648276 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:42.649236 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:42.650656 (+ 1420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:42.652610 (+ 1954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:42.653262 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:42.653466 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:42.657494 (+ 4028us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.24.2:48713: BlockingRecv error: recv got EOF from 127.8.24.2:48713 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 08 16:52:42 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.3@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:52:42 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.3@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 16:52:42.778443 22844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:42.774381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:43045 (local address 127.8.24.3:50783)
0808 16:52:42.774523 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:42.774533 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:42.774589 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:42.774702 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:42.774702 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:42.774858 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:42.774906 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:42.774907 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:42.775474 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:42.775474 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:42.775999 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:42.776000 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:42.776277 (+ 277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:42.776278 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:42.776345 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:42.776782 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:42.776787 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:42.778364 (+ 1577us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.24.62:43045: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":85,"threads_started":1}
W20260808 16:52:42.778525 22839 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:52:42.774423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:50783 (local address 127.8.24.62:43045)
0808 16:52:42.774482 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:42.774484 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:42.774584 (+ 100us) server_negotiation.cc:407] Connection header received
0808 16:52:42.774640 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:42.774641 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:42.774654 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:42.774691 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:42.774952 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:42.775423 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:42.776049 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:42.776219 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:42.776259 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:42.778463 (+ 2204us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.24.3:50783: BlockingRecv error: recv got EOF from 127.8.24.3:50783 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
Aug 08 16:52:43 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.1@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:52:43 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.1@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 16:52:43.522837 22846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:43.518674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:43045 (local address 127.8.24.1:34073)
0808 16:52:43.518797 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:43.518808 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:43.518857 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:43.518963 (+ 106us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:43.518964 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:43.519131 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:43.519182 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:43.519184 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:43.519814 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:43.519815 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:43.520361 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:43.520364 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:43.520692 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:43.520692 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:43.520732 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:43.521169 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:43.521173 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:43.522771 (+ 1598us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.24.62:43045: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":66,"threads_started":1}
W20260808 16:52:43.522922 22849 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:52:43.518740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:34073 (local address 127.8.24.62:43045)
0808 16:52:43.518852 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:43.518854 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:43.518863 (+ 9us) server_negotiation.cc:407] Connection header received
0808 16:52:43.518893 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:43.518893 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:43.518906 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:43.518958 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:43.519240 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:43.519756 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:43.520436 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:43.520639 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:43.520686 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:43.522854 (+ 2168us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.24.1:34073: BlockingRecv error: recv got EOF from 127.8.24.1:34073 (error 108)
Metrics: {"server-negotiator.queue_time_us":92,"thread_start_us":64,"threads_started":1}
Aug 08 16:52:43 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.2@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:52:43 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.2@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 16:52:43.665149 22850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:43.657915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:43045 (local address 127.8.24.2:46431)
0808 16:52:43.658037 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:43.658047 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:43.658101 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:43.658209 (+ 108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:43.658210 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:43.658372 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:43.658419 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:43.658421 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:43.659003 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:43.659003 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:43.659514 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:43.659516 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:43.659780 (+ 264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:43.659780 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:43.659814 (+ 34us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:43.663444 (+ 3630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:43.663449 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:43.665061 (+ 1612us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.24.62:43045: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":71,"threads_started":1}
W20260808 16:52:43.665192 22849 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:52:43.657963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:46431 (local address 127.8.24.62:43045)
0808 16:52:43.658034 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:43.658036 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:43.658090 (+ 54us) server_negotiation.cc:407] Connection header received
0808 16:52:43.658152 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:43.658152 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:43.658164 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:43.658199 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:43.658470 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:43.658949 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:43.659561 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:43.659736 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:43.659778 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:43.665139 (+ 5361us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.24.2:46431: BlockingRecv error: recv got EOF from 127.8.24.2:46431 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 08 16:52:43 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.3@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:52:43 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.3@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 16:52:43.783236 22851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:43.778858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:43045 (local address 127.8.24.3:54411)
0808 16:52:43.778991 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:43.779003 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:43.779066 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:43.779207 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:43.779208 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:43.779369 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:43.779416 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:43.779418 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:43.780040 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:43.780041 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:43.780552 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:43.780554 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:43.780932 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:43.780933 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:43.780982 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:43.781470 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:43.781474 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:43.783177 (+ 1703us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.24.62:43045: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":79,"threads_started":1}
W20260808 16:52:43.783329 22849 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:52:43.778933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:54411 (local address 127.8.24.62:43045)
0808 16:52:43.778994 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:43.778997 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:43.779054 (+ 57us) server_negotiation.cc:407] Connection header received
0808 16:52:43.779131 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:43.779132 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:43.779145 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:43.779185 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:43.779480 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:43.779972 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:43.780621 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:43.780840 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:43.780901 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:43.783269 (+ 2368us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.24.3:54411: BlockingRecv error: recv got EOF from 127.8.24.3:54411 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Aug 08 16:52:44 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.1@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:52:44 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.1@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 16:52:44.527549 22853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:44.523279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:43045 (local address 127.8.24.1:56421)
0808 16:52:44.523414 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:44.523424 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:44.523484 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:44.523648 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:44.523649 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:44.523810 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:44.523860 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:44.523862 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:44.524539 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:44.524540 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:44.525093 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:44.525095 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:44.525387 (+ 292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:44.525387 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:44.525424 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:44.525822 (+ 398us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:44.525826 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:44.527478 (+ 1652us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.24.62:43045: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":69,"threads_started":1}
W20260808 16:52:44.527635 22854 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:52:44.523342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:56421 (local address 127.8.24.62:43045)
0808 16:52:44.523445 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:44.523447 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:44.523468 (+ 21us) server_negotiation.cc:407] Connection header received
0808 16:52:44.523536 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:44.523537 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:44.523547 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:44.523594 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:44.523959 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:44.524471 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:44.525151 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:44.525339 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:44.525381 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:44.527566 (+ 2185us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.24.1:56421: BlockingRecv error: recv got EOF from 127.8.24.1:56421 (error 108)
Metrics: {"server-negotiator.queue_time_us":82,"thread_start_us":54,"threads_started":1}
Aug 08 16:52:44 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.2@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:52:44 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.2@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 16:52:44.669943 22856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:44.665581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:43045 (local address 127.8.24.2:34061)
0808 16:52:44.665720 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:44.665733 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:44.665793 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:44.665931 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:44.665931 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:44.666095 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:44.666146 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:44.666148 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:44.666807 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:44.666809 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:44.667341 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:44.667344 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:44.667669 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:44.667670 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:44.667723 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:44.668139 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:44.668143 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:44.669873 (+ 1730us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.24.62:43045: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":64,"threads_started":1}
W20260808 16:52:44.670056 22854 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:52:44.665702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:34061 (local address 127.8.24.62:43045)
0808 16:52:44.665768 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:44.665770 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:44.665779 (+ 9us) server_negotiation.cc:407] Connection header received
0808 16:52:44.665856 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:44.665857 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:44.665869 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:44.665915 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:44.666218 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:44.666737 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:44.667421 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:44.667609 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:44.667664 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:44.669988 (+ 2324us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.24.2:34061: BlockingRecv error: recv got EOF from 127.8.24.2:34061 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 08 16:52:44 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.3@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:52:44 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.3@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 16:52:44.787843 22857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:44.783634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:43045 (local address 127.8.24.3:60135)
0808 16:52:44.783782 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:44.783791 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:44.783846 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:44.783961 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:44.783962 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:44.784118 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:44.784166 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:44.784167 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:44.784761 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:44.784762 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:44.785288 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:44.785289 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:44.785576 (+ 287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:44.785576 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:44.785611 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:44.786122 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:44.786128 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:44.787789 (+ 1661us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.24.62:43045: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":78,"threads_started":1}
W20260808 16:52:44.787936 22854 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:52:44.783694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:60135 (local address 127.8.24.62:43045)
0808 16:52:44.783756 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:44.783758 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:44.783840 (+ 82us) server_negotiation.cc:407] Connection header received
0808 16:52:44.783888 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:44.783889 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:44.783901 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:44.783949 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:44.784213 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:44.784698 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:44.785349 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:44.785527 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:44.785572 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:44.787878 (+ 2306us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.24.3:60135: BlockingRecv error: recv got EOF from 127.8.24.3:60135 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 08 16:52:45 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.1@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:52:45 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.1@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 16:52:45.532311 22858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:45.527946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:43045 (local address 127.8.24.1:38933)
0808 16:52:45.528082 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:45.528095 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:45.528154 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:45.528312 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:45.528313 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:45.528491 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:45.528539 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:45.528541 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:45.529226 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:45.529228 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:45.529749 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:45.529751 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:45.530089 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:45.530090 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:45.530152 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:45.530609 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:45.530613 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:45.532255 (+ 1642us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.24.62:43045: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":68,"threads_started":1}
W20260808 16:52:45.532415 22860 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:52:45.528017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:38933 (local address 127.8.24.62:43045)
0808 16:52:45.528147 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:45.528149 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:45.528157 (+ 8us) server_negotiation.cc:407] Connection header received
0808 16:52:45.528202 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:45.528202 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:45.528212 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:45.528267 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:45.528657 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:45.529149 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:45.529828 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:45.530029 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:45.530084 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:45.532339 (+ 2255us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.24.1:38933: BlockingRecv error: recv got EOF from 127.8.24.1:38933 (error 108)
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":71,"threads_started":1}
Aug 08 16:52:45 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.2@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:52:45 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.2@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 16:52:45.674677 22862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:45.670356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:43045 (local address 127.8.24.2:53605)
0808 16:52:45.670493 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:45.670504 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:45.670567 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:45.670695 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:45.670695 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:45.670867 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:45.670914 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:45.670915 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:45.671549 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:45.671550 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:45.672074 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:45.672076 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:45.672417 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:45.672417 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:45.672475 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:45.672860 (+ 385us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:45.672864 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:45.674620 (+ 1756us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.24.62:43045: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":79,"threads_started":1}
W20260808 16:52:45.674762 22860 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:52:45.670425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:53605 (local address 127.8.24.62:43045)
0808 16:52:45.670485 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:45.670487 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:45.670556 (+ 69us) server_negotiation.cc:407] Connection header received
0808 16:52:45.670627 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:45.670628 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:45.670641 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:45.670689 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:45.670979 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:45.671478 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:45.672144 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:45.672356 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:45.672410 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:45.674704 (+ 2294us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.24.2:53605: BlockingRecv error: recv got EOF from 127.8.24.2:53605 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Aug 08 16:52:45 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.3@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:52:45 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.3@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 16:52:45.792532 22863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:45.788248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:43045 (local address 127.8.24.3:48717)
0808 16:52:45.788367 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:45.788379 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:45.788443 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:45.788574 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:45.788575 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:45.788739 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:45.788781 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:45.788782 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:45.789419 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:45.789421 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:45.789933 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:45.789935 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:45.790248 (+ 313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:45.790249 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:45.790302 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:45.790759 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:45.790763 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:45.792456 (+ 1693us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.24.62:43045: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":66,"threads_started":1}
W20260808 16:52:45.792608 22860 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:52:45.788311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:48717 (local address 127.8.24.62:43045)
0808 16:52:45.788371 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:45.788374 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:45.788434 (+ 60us) server_negotiation.cc:407] Connection header received
0808 16:52:45.788500 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:45.788501 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:45.788513 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:45.788566 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:45.788857 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:45.789351 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:45.790005 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:45.790191 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:45.790244 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:45.792545 (+ 2301us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.24.3:48717: BlockingRecv error: recv got EOF from 127.8.24.3:48717 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Aug 08 16:52:46 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.1@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:52:46 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.1@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 16:52:46.536928 22865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:46.532764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:43045 (local address 127.8.24.1:46297)
0808 16:52:46.532894 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:46.532904 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:46.532952 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:46.533066 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:46.533067 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:46.533215 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:46.533265 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:46.533266 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:46.533865 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:46.533866 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:46.534403 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:46.534405 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:46.534846 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:46.534847 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:46.534886 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:46.535310 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:46.535314 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:46.536861 (+ 1547us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.24.62:43045: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":66,"threads_started":1}
W20260808 16:52:46.537036 22866 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:52:46.532823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:46297 (local address 127.8.24.62:43045)
0808 16:52:46.532942 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:46.532943 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:46.532952 (+ 9us) server_negotiation.cc:407] Connection header received
0808 16:52:46.532994 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:46.532995 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:46.533006 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:46.533061 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:46.533313 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:46.533811 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:46.534579 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:46.534783 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:46.534843 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:46.536962 (+ 2119us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.24.1:46297: BlockingRecv error: recv got EOF from 127.8.24.1:46297 (error 108)
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":56,"threads_started":1}
Aug 08 16:52:46 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.2@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:52:46 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.2@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 16:52:46.679241 22867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:46.675064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:43045 (local address 127.8.24.2:33663)
0808 16:52:46.675182 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:46.675192 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:46.675249 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:46.675358 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:46.675359 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:46.675505 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:46.675553 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:46.675554 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:46.676170 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:46.676170 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:46.676696 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:46.676697 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:46.676988 (+ 291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:46.676989 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:46.677024 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:46.677419 (+ 395us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:46.677424 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:46.679174 (+ 1750us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.24.62:43045: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":55,"threads_started":1}
W20260808 16:52:46.679323 22866 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:52:46.675143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:33663 (local address 127.8.24.62:43045)
0808 16:52:46.675192 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:46.675194 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:46.675239 (+ 45us) server_negotiation.cc:407] Connection header received
0808 16:52:46.675291 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:46.675292 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:46.675302 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:46.675347 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:46.675609 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:46.676088 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:46.676755 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:46.676942 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:46.676986 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:46.679266 (+ 2280us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.24.2:33663: BlockingRecv error: recv got EOF from 127.8.24.2:33663 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
Aug 08 16:52:46 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.3@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:52:46 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.3@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 16:52:46.796973 22868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:46.792910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:43045 (local address 127.8.24.3:56001)
0808 16:52:46.793024 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:46.793036 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:46.793100 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:46.793222 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:46.793223 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:46.793382 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:46.793426 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:46.793428 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:46.794107 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:46.794108 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:46.794688 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:46.794690 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:46.795015 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:46.795016 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:46.795075 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:46.795460 (+ 385us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:46.795464 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:46.796901 (+ 1437us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.24.62:43045: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":56,"threads_started":1}
W20260808 16:52:46.797049 22866 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:52:46.792976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:56001 (local address 127.8.24.62:43045)
0808 16:52:46.793027 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:46.793030 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:46.793080 (+ 50us) server_negotiation.cc:407] Connection header received
0808 16:52:46.793148 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:46.793148 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:46.793162 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:46.793210 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:46.793490 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:46.794031 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:46.794757 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:46.794958 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:46.795012 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:46.796996 (+ 1984us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.24.3:56001: BlockingRecv error: recv got EOF from 127.8.24.3:56001 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
Aug 08 16:52:47 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.1@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:52:47 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.1@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 16:52:47.541517 22869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:47.537325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:43045 (local address 127.8.24.1:42995)
0808 16:52:47.537467 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:47.537477 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:47.537522 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:47.537675 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:47.537676 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:47.537824 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:47.537872 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:47.537873 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:47.538541 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:47.538542 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:47.539072 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:47.539073 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:47.539357 (+ 284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:47.539357 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:47.539392 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:47.539817 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:47.539821 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:47.541461 (+ 1640us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.24.62:43045: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":66,"threads_started":1}
W20260808 16:52:47.541621 22871 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:52:47.537432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:42995 (local address 127.8.24.62:43045)
0808 16:52:47.537557 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:47.537558 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:47.537567 (+ 9us) server_negotiation.cc:407] Connection header received
0808 16:52:47.537587 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:47.537587 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:47.537599 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:47.537656 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:47.537921 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:47.538464 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:47.539128 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:47.539311 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:47.539354 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:47.541549 (+ 2195us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.24.1:42995: BlockingRecv error: recv got EOF from 127.8.24.1:42995 (error 108)
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":58,"threads_started":1}
Aug 08 16:52:47 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.2@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:52:47 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.2@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 16:52:47.684093 22873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:47.679652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:43045 (local address 127.8.24.2:55825)
0808 16:52:47.679809 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:47.679821 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:47.679877 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:47.679998 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:47.679999 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:47.680152 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:47.680201 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:47.680202 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:47.680950 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:47.680950 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:47.681481 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:47.681483 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:47.681866 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:47.681866 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:47.681919 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:47.682375 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:47.682380 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:47.684034 (+ 1654us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.24.62:43045: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":96,"threads_started":1}
W20260808 16:52:47.684201 22871 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:52:47.679723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:55825 (local address 127.8.24.62:43045)
0808 16:52:47.679782 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:47.679784 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:47.679870 (+ 86us) server_negotiation.cc:407] Connection header received
0808 16:52:47.679921 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:47.679922 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:47.679933 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:47.679981 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:47.680267 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:47.680888 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:47.681585 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:47.681788 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:47.681856 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:47.684139 (+ 2283us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.24.2:55825: BlockingRecv error: recv got EOF from 127.8.24.2:55825 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Aug 08 16:52:47 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.3@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:52:47 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.3@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 16:52:47.801614 22876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:47.797421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:43045 (local address 127.8.24.3:38635)
0808 16:52:47.797534 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:47.797543 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:47.797589 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:47.797703 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:47.797704 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:47.797857 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:47.797903 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:47.797905 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:47.798577 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:47.798577 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:47.799093 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:47.799095 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:47.799378 (+ 283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:47.799379 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:47.799414 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:47.799803 (+ 389us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:47.799807 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:47.801530 (+ 1723us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.24.62:43045: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":65,"threads_started":1}
W20260808 16:52:47.801673 22871 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:52:47.797505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:38635 (local address 127.8.24.62:43045)
0808 16:52:47.797575 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:47.797576 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:47.797585 (+ 9us) server_negotiation.cc:407] Connection header received
0808 16:52:47.797637 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:47.797637 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:47.797650 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:47.797692 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:47.797966 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:47.798515 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:47.799149 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:47.799332 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:47.799376 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:47.801620 (+ 2244us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.24.3:38635: BlockingRecv error: recv got EOF from 127.8.24.3:38635 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 08 16:52:48 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.1@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:52:48 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.1@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 16:52:48.546305 22879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:48.541960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:43045 (local address 127.8.24.1:33903)
0808 16:52:48.542111 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:48.542123 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:48.542194 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:48.542360 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:48.542361 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:48.542535 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:48.542587 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:48.542589 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:48.543203 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:48.543204 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:48.543735 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:48.543736 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:48.544022 (+ 286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:48.544023 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:48.544065 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:48.544512 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:48.544517 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:48.546249 (+ 1732us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.24.62:43045: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":72,"threads_started":1}
W20260808 16:52:48.546427 22882 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:52:48.542016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:33903 (local address 127.8.24.62:43045)
0808 16:52:48.542115 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:48.542117 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:48.542180 (+ 63us) server_negotiation.cc:407] Connection header received
0808 16:52:48.542246 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:48.542247 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:48.542260 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:48.542322 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:48.542650 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:48.543133 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:48.543785 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:48.543960 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:48.544001 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:48.546331 (+ 2330us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.24.1:33903: BlockingRecv error: recv got EOF from 127.8.24.1:33903 (error 108)
Metrics: {"server-negotiator.queue_time_us":77,"thread_start_us":49,"threads_started":1}
Aug 08 16:52:48 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.2@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:52:48 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.2@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 16:52:48.691733 22887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:48.684555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:43045 (local address 127.8.24.2:39617)
0808 16:52:48.684689 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:48.684699 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:48.684753 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:48.684870 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:48.684871 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:48.685024 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:48.685071 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:48.685072 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:48.685683 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:48.685684 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:48.686194 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:48.686195 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:48.686546 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:48.686548 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:48.686602 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:48.689711 (+ 3109us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:48.689716 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:48.691674 (+ 1958us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.24.62:43045: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":57,"threads_started":1}
W20260808 16:52:48.691819 22882 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:52:48.684627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:39617 (local address 127.8.24.62:43045)
0808 16:52:48.684682 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:48.684684 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:48.684749 (+ 65us) server_negotiation.cc:407] Connection header received
0808 16:52:48.684796 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:48.684796 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:48.684808 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:48.684861 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:48.685119 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:48.685621 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:48.686250 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:48.686461 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:48.686513 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:48.691757 (+ 5244us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.24.2:39617: BlockingRecv error: recv got EOF from 127.8.24.2:39617 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Aug 08 16:52:48 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.3@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:52:48 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.3@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 16:52:48.806260 22888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:48.802055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:43045 (local address 127.8.24.3:44331)
0808 16:52:48.802198 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:48.802210 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:48.802278 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:48.802431 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:48.802431 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:48.802591 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:48.802635 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:48.802637 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:48.803248 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:48.803249 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:48.803770 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:48.803772 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:48.804083 (+ 311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:48.804084 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:48.804136 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:48.804546 (+ 410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:48.804550 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:48.806185 (+ 1635us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.24.62:43045: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":84,"threads_started":1}
W20260808 16:52:48.806443 22882 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:52:48.802158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:44331 (local address 127.8.24.62:43045)
0808 16:52:48.802223 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:48.802225 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:48.802262 (+ 37us) server_negotiation.cc:407] Connection header received
0808 16:52:48.802355 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:48.802356 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:48.802371 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:48.802417 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:48.802701 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:48.803177 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:48.803840 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:48.804025 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:48.804079 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:48.806289 (+ 2210us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.24.3:44331: BlockingRecv error: recv got EOF from 127.8.24.3:44331 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Aug 08 16:52:49 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.1@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:52:49 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.1@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 16:52:49.550974 22889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:49.546732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:43045 (local address 127.8.24.1:51485)
0808 16:52:49.546869 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:49.546880 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:49.546938 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:49.547054 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:49.547054 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:49.547206 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:49.547264 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:49.547265 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:49.547901 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:49.547902 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:49.548525 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:49.548527 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:49.548838 (+ 311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:49.548840 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:49.548903 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:49.549313 (+ 410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:49.549318 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:49.550901 (+ 1583us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.24.62:43045: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":70,"threads_started":1}
W20260808 16:52:49.551062 22890 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:52:49.546784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:51485 (local address 127.8.24.62:43045)
0808 16:52:49.546909 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:49.546911 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:49.546921 (+ 10us) server_negotiation.cc:407] Connection header received
0808 16:52:49.546983 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:49.546984 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:49.546993 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:49.547043 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:49.547312 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:49.547837 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:49.548576 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:49.548764 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:49.548822 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:49.550999 (+ 2177us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.24.1:51485: BlockingRecv error: recv got EOF from 127.8.24.1:51485 (error 108)
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":62,"threads_started":1}
Aug 08 16:52:49 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.2@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:52:49 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.2@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 16:52:49.696295 22892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:49.692154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:43045 (local address 127.8.24.2:51945)
0808 16:52:49.692278 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:49.692287 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:49.692343 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:49.692460 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:49.692461 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:49.692609 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:49.692656 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:49.692657 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:49.693235 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:49.693236 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:49.693746 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:49.693747 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:49.694150 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:49.694152 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:49.694231 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:49.694658 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:49.694662 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:49.696236 (+ 1574us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.24.62:43045: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":64,"threads_started":1}
W20260808 16:52:49.696380 22890 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:52:49.692208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:51945 (local address 127.8.24.62:43045)
0808 16:52:49.692262 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:49.692263 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:49.692335 (+ 72us) server_negotiation.cc:407] Connection header received
0808 16:52:49.692396 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:49.692396 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:49.692410 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:49.692446 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:49.692701 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:49.693181 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:49.693810 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:49.694077 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:49.694139 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:49.696322 (+ 2183us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.24.2:51945: BlockingRecv error: recv got EOF from 127.8.24.2:51945 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
Aug 08 16:52:49 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.3@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:52:49 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.3@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 16:52:49.810761 22893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:49.806703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:43045 (local address 127.8.24.3:33677)
0808 16:52:49.806854 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:49.806866 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:49.806921 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:49.807036 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:49.807037 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:49.807185 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:49.807230 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:49.807232 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:49.807831 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:49.807831 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:49.808337 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:49.808338 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:49.808631 (+ 293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:49.808631 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:49.808672 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:49.809053 (+ 381us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:49.809057 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:49.810706 (+ 1649us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.24.62:43045: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":86,"threads_started":1}
W20260808 16:52:49.810842 22890 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:52:49.806827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:33677 (local address 127.8.24.62:43045)
0808 16:52:49.806882 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:49.806884 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:49.806906 (+ 22us) server_negotiation.cc:407] Connection header received
0808 16:52:49.806965 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:49.806966 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:49.806978 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:49.807023 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:49.807293 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:49.807770 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:49.808401 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:49.808579 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:49.808629 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:49.810782 (+ 2153us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.24.3:33677: BlockingRecv error: recv got EOF from 127.8.24.3:33677 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Aug 08 16:52:50 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.1@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:52:50 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.1@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 16:52:50.555720 22894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:50.551402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:43045 (local address 127.8.24.1:33869)
0808 16:52:50.551529 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:50.551539 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:50.551592 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:50.551719 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:50.551720 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:50.551882 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:50.551931 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:50.551932 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:50.552700 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:50.552701 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:50.553233 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:50.553235 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:50.553520 (+ 285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:50.553521 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:50.553560 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:50.553953 (+ 393us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:50.553957 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:50.555662 (+ 1705us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.24.62:43045: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":63,"threads_started":1}
W20260808 16:52:50.555810 22895 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:52:50.551463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:33869 (local address 127.8.24.62:43045)
0808 16:52:50.551581 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:50.551583 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:50.551591 (+ 8us) server_negotiation.cc:407] Connection header received
0808 16:52:50.551635 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:50.551635 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:50.551646 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:50.551708 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:50.551990 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:50.552636 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:50.553291 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:50.553474 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:50.553514 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:50.555747 (+ 2233us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.24.1:33869: BlockingRecv error: recv got EOF from 127.8.24.1:33869 (error 108)
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":63,"threads_started":1}
Aug 08 16:52:50 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.2@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:52:50 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.2@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 16:52:50.700732 22896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:50.696723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:43045 (local address 127.8.24.2:53051)
0808 16:52:50.696861 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:50.696870 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:50.696928 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:50.697039 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:50.697040 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:50.697206 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:50.697250 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:50.697252 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:50.697840 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:50.697840 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:50.698387 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:50.698389 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:50.698679 (+ 290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:50.698680 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:50.698719 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:50.699121 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:50.699126 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:50.700679 (+ 1553us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.24.62:43045: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":76,"threads_started":1}
W20260808 16:52:50.700812 22895 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:52:50.696793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:53051 (local address 127.8.24.62:43045)
0808 16:52:50.696851 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:50.696853 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:50.696921 (+ 68us) server_negotiation.cc:407] Connection header received
0808 16:52:50.696970 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:50.696971 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:50.696981 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:50.697026 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:50.697309 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:50.697785 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:50.698441 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:50.698620 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:50.698666 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:50.700754 (+ 2088us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.24.2:53051: BlockingRecv error: recv got EOF from 127.8.24.2:53051 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Aug 08 16:52:50 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.3@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:52:50 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.3@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 16:52:50.815304 22900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:50.811156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:43045 (local address 127.8.24.3:50343)
0808 16:52:50.811304 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:50.811314 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:50.811378 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:50.811505 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:50.811505 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:50.811669 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:50.811713 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:50.811715 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:50.812341 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:50.812342 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:50.812860 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:50.812862 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:50.813180 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:50.813181 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:50.813237 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:50.813623 (+ 386us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:50.813627 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:50.815233 (+ 1606us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.24.62:43045: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":77,"threads_started":1}
W20260808 16:52:50.815382 22895 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:52:50.811219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:50343 (local address 127.8.24.62:43045)
0808 16:52:50.811275 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:50.811277 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:50.811372 (+ 95us) server_negotiation.cc:407] Connection header received
0808 16:52:50.811427 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:50.811428 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:50.811443 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:50.811492 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:50.811783 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:50.812268 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:50.812935 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:50.813121 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:50.813175 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:50.815327 (+ 2152us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.24.3:50343: BlockingRecv error: recv got EOF from 127.8.24.3:50343 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Aug 08 16:52:51 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.1@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:52:51 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.1@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 16:52:51.560416 22903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:51.556107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:43045 (local address 127.8.24.1:38411)
0808 16:52:51.556257 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:51.556268 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:51.556324 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:51.556466 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:51.556467 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:51.556659 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:51.556718 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:51.556721 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:51.557466 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:51.557467 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:51.557995 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:51.557997 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:51.558279 (+ 282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:51.558279 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:51.558316 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:51.558784 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:51.558788 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:51.560355 (+ 1567us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.24.62:43045: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":75,"threads_started":1}
W20260808 16:52:51.560509 22904 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:52:51.556237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:38411 (local address 127.8.24.62:43045)
0808 16:52:51.556359 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:51.556361 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:51.556368 (+ 7us) server_negotiation.cc:407] Connection header received
0808 16:52:51.556389 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:51.556389 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:51.556399 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:51.556456 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:51.556809 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:51.557399 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:51.558053 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:51.558233 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:51.558275 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:51.560443 (+ 2168us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.24.1:38411: BlockingRecv error: recv got EOF from 127.8.24.1:38411 (error 108)
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":69,"threads_started":1}
Aug 08 16:52:51 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.2@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:52:51 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.2@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 16:52:51.705199 22905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:51.701152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:43045 (local address 127.8.24.2:54561)
0808 16:52:51.701295 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:51.701304 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:51.701360 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:51.701474 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:51.701474 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:51.701626 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:51.701674 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:51.701675 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:51.702261 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:51.702262 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:51.702800 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:51.702801 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:51.703072 (+ 271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:51.703073 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:51.703108 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:51.703522 (+ 414us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:51.703526 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:51.705122 (+ 1596us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.24.62:43045: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":79,"threads_started":1}
W20260808 16:52:51.705412 22904 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:52:51.701227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:54561 (local address 127.8.24.62:43045)
0808 16:52:51.701282 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:51.701284 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:51.701356 (+ 72us) server_negotiation.cc:407] Connection header received
0808 16:52:51.701403 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:51.701404 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:51.701416 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:51.701463 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:51.701722 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:51.702210 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:51.702855 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:51.703029 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:51.703069 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:51.705229 (+ 2160us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.24.2:54561: BlockingRecv error: recv got EOF from 127.8.24.2:54561 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Aug 08 16:52:51 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.3@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:52:51 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.3@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 16:52:51.819695 22908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:51.815712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:43045 (local address 127.8.24.3:51351)
0808 16:52:51.815819 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:51.815828 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:51.815881 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:51.815987 (+ 106us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:51.815988 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:51.816152 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:51.816201 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:51.816202 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:51.816775 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:51.816776 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:51.817308 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:51.817310 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:51.817576 (+ 266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:51.817576 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:51.817611 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:51.817989 (+ 378us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:51.817993 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:51.819637 (+ 1644us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.24.62:43045: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":50,"threads_started":1}
W20260808 16:52:51.819788 22904 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:52:51.815765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:51351 (local address 127.8.24.62:43045)
0808 16:52:51.815803 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:51.815806 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:51.815876 (+ 70us) server_negotiation.cc:407] Connection header received
0808 16:52:51.815920 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:51.815921 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:51.815932 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:51.815977 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:51.816247 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:51.816718 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:51.817364 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:51.817532 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:51.817574 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:51.819721 (+ 2147us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.24.3:51351: BlockingRecv error: recv got EOF from 127.8.24.3:51351 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
Aug 08 16:52:52 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.1@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:52:52 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.1@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 16:52:52.565088 22909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:52.560827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:43045 (local address 127.8.24.1:43049)
0808 16:52:52.560965 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:52.560977 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:52.561037 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:52.561206 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:52.561208 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:52.561367 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:52.561418 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:52.561419 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:52.562026 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:52.562026 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:52.562678 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:52.562681 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:52.562999 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:52.562999 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:52.563035 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:52.563458 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:52.563463 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:52.565031 (+ 1568us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.24.62:43045: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":63,"threads_started":1}
W20260808 16:52:52.565179 22910 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:52:52.560890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:43049 (local address 127.8.24.62:43045)
0808 16:52:52.561010 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:52.561012 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:52.561020 (+ 8us) server_negotiation.cc:407] Connection header received
0808 16:52:52.561086 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:52.561087 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:52.561108 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:52.561163 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:52.561482 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:52.561969 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:52.562749 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:52.562946 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:52.562996 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:52.565113 (+ 2117us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.24.1:43049: BlockingRecv error: recv got EOF from 127.8.24.1:43049 (error 108)
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":67,"threads_started":1}
Aug 08 16:52:52 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.2@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:52:52 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.2@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 16:52:52.709805 22911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:52.705682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:43045 (local address 127.8.24.2:37851)
0808 16:52:52.705811 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:52.705821 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:52.705879 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:52.706018 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:52.706019 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:52.706190 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:52.706235 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:52.706237 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:52.706900 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:52.706902 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:52.707422 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:52.707424 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:52.707747 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:52.707748 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:52.707801 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:52.708194 (+ 393us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:52.708198 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:52.709732 (+ 1534us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.24.62:43045: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":62,"threads_started":1}
W20260808 16:52:52.709880 22910 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:52:52.705807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:37851 (local address 127.8.24.62:43045)
0808 16:52:52.705861 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:52.705863 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:52.705871 (+ 8us) server_negotiation.cc:407] Connection header received
0808 16:52:52.705944 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:52.705944 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:52.705957 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:52.706003 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:52.706308 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:52.706827 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:52.707493 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:52.707690 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:52.707743 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:52.709829 (+ 2086us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.24.2:37851: BlockingRecv error: recv got EOF from 127.8.24.2:37851 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Aug 08 16:52:52 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.3@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:52:52 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.3@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 16:52:52.824105 22912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:52.820079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:43045 (local address 127.8.24.3:38715)
0808 16:52:52.820194 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:52.820226 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:52.820281 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:52.820393 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:52.820394 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:52.820551 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:52.820598 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:52.820600 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:52.821190 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:52.821190 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:52.821725 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:52.821727 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:52.821997 (+ 270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:52.821997 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:52.822036 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:52.822470 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:52.822474 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:52.824035 (+ 1561us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.24.62:43045: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":56,"threads_started":1}
W20260808 16:52:52.824170 22910 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:52:52.820129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:38715 (local address 127.8.24.62:43045)
0808 16:52:52.820177 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:52.820179 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:52.820271 (+ 92us) server_negotiation.cc:407] Connection header received
0808 16:52:52.820323 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:52.820324 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:52.820336 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:52.820381 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:52.820646 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:52.821134 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:52.821782 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:52.821951 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:52.821992 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:52.824120 (+ 2128us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.24.3:38715: BlockingRecv error: recv got EOF from 127.8.24.3:38715 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Aug 08 16:52:53 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.1@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:52:53 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.1@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 16:52:53.569532 22913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:53.565544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:43045 (local address 127.8.24.1:44665)
0808 16:52:53.565690 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:53.565703 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:53.565761 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:53.565893 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:53.565894 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:53.566041 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:53.566087 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:53.566089 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:53.566745 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:53.566746 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:53.567259 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:53.567260 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:53.567538 (+ 278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:53.567538 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:53.567573 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:53.567957 (+ 384us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:53.567960 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:53.569476 (+ 1516us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.24.62:43045: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":77,"threads_started":1}
W20260808 16:52:53.569628 22915 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:52:53.565617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:44665 (local address 127.8.24.62:43045)
0808 16:52:53.565745 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:53.565747 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:53.565756 (+ 9us) server_negotiation.cc:407] Connection header received
0808 16:52:53.565811 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:53.565812 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:53.565822 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:53.565879 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:53.566172 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:53.566688 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:53.567315 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:53.567495 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:53.567535 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:53.569561 (+ 2026us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.24.1:44665: BlockingRecv error: recv got EOF from 127.8.24.1:44665 (error 108)
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":64,"threads_started":1}
Aug 08 16:52:53 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.2@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:52:53 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.2@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 16:52:53.717199 22916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:53.710201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:43045 (local address 127.8.24.2:34307)
0808 16:52:53.710305 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:53.710315 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:53.710380 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:53.710501 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:53.710502 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:53.710642 (+ 140us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:53.710688 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:53.710690 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:53.711256 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:53.711256 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:53.711818 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:53.711821 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:53.712240 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:53.712241 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:53.712285 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:53.715565 (+ 3280us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:53.715569 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:53.717136 (+ 1567us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.24.62:43045: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":83,"thread_start_us":50,"threads_started":1}
W20260808 16:52:53.717281 22915 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:52:53.710271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:34307 (local address 127.8.24.62:43045)
0808 16:52:53.710334 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:53.710356 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:53.710369 (+ 13us) server_negotiation.cc:407] Connection header received
0808 16:52:53.710434 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:53.710435 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:53.710445 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:53.710488 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:53.710736 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:53.711205 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:53.711874 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:53.712160 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:53.712228 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:53.717226 (+ 4998us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.24.2:34307: BlockingRecv error: recv got EOF from 127.8.24.2:34307 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 08 16:52:53 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.3@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:52:53 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.3@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 16:52:53.828688 22917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:53.824506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:43045 (local address 127.8.24.3:36137)
0808 16:52:53.824628 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:53.824639 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:53.824703 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:53.824845 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:53.824845 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:53.825004 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:53.825047 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:53.825049 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:53.825679 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:53.825680 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:53.826190 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:53.826192 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:53.826554 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:53.826555 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:53.826609 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:53.826985 (+ 376us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:53.826989 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:53.828614 (+ 1625us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.24.62:43045: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":63,"threads_started":1}
W20260808 16:52:53.828751 22915 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:52:53.824577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:36137 (local address 127.8.24.62:43045)
0808 16:52:53.824632 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:53.824634 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:53.824695 (+ 61us) server_negotiation.cc:407] Connection header received
0808 16:52:53.824770 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:53.824770 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:53.824785 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:53.824833 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:53.825117 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:53.825607 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:53.826270 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:53.826494 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:53.826549 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:53.828690 (+ 2141us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.24.3:36137: BlockingRecv error: recv got EOF from 127.8.24.3:36137 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Aug 08 16:52:54 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.1@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:52:54 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.1@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 16:52:54.574083 22918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:54.570010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:43045 (local address 127.8.24.1:49013)
0808 16:52:54.570127 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:54.570137 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:54.570193 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:54.570318 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:54.570319 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:54.570500 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:54.570550 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:54.570552 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:54.571137 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:54.571137 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:54.571660 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:54.571661 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:54.571941 (+ 280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:54.571941 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:54.571976 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:54.572426 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:54.572430 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:54.574026 (+ 1596us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.24.62:43045: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":55,"threads_started":1}
W20260808 16:52:54.574179 22919 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:52:54.570070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:49013 (local address 127.8.24.62:43045)
0808 16:52:54.570181 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:54.570183 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:54.570192 (+ 9us) server_negotiation.cc:407] Connection header received
0808 16:52:54.570240 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:54.570240 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:54.570251 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:54.570310 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:54.570603 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:54.571083 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:54.571713 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:54.571895 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:54.571938 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:54.574113 (+ 2175us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.24.1:49013: BlockingRecv error: recv got EOF from 127.8.24.1:49013 (error 108)
Metrics: {"server-negotiator.queue_time_us":92,"thread_start_us":56,"threads_started":1}
Aug 08 16:52:54 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.2@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:52:54 dist-test-slave-6zbq krb5kdc[21210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.2@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 16:52:54.721935 22921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:54.717732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:43045 (local address 127.8.24.2:40055)
0808 16:52:54.717865 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:54.717876 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:54.717936 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:54.718070 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:54.718071 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:54.718232 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:54.718282 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:54.718284 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:54.718920 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:54.718921 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:54.719459 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:54.719461 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:54.719748 (+ 287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:54.719748 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:54.719783 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:54.720200 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:54.720204 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:54.721858 (+ 1654us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.24.62:43045: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":84,"threads_started":1}
W20260808 16:52:54.722007 22919 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:52:54.717782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:40055 (local address 127.8.24.62:43045)
0808 16:52:54.717836 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:54.717838 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:54.717926 (+ 88us) server_negotiation.cc:407] Connection header received
0808 16:52:54.717990 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:54.717991 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:54.718006 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:54.718057 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:54.718363 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:54.718856 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:54.719518 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:54.719698 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:54.719745 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:54.721955 (+ 2210us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.24.2:40055: BlockingRecv error: recv got EOF from 127.8.24.2:40055 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 16:52:54.749750 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 22134
I20260808 16:52:54.759346 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 22398
I20260808 16:52:54.775583 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 22612
I20260808 16:52:54.786098 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 22021
2026-08-08T16:52:54Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17581 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-08T16:52:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T16:52:54Z Disabled control of system clock
I20260808 16:52:54.817409 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786207919384338-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786207919384338-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786207919384338-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:40631
--webserver_interface=127.8.24.62
--webserver_port=0
--builtin_ntp_servers=127.8.24.20:38231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:40631
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260808 16:52:54.896222 22927 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:52:54.896394 22927 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:52:54.896411 22927 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:52:54.897485 22927 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 16:52:54.897516 22927 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:52:54.897527 22927 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:52:54.897538 22927 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 16:52:54.897548 22927 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 16:52:54.898869 22927 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:38231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786207919384338-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786207919384338-8288-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:40631
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:40631
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.24.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22927
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786207919384338-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:52:54.899052 22927 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:52:54.899283 22927 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:52:54.903915 22937 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:54.903959 22935 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:54.903918 22941 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:54.903970 22927 server_base.cc:1034] running on GCE node
I20260808 16:52:54.904340 22927 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:52:54.904588 22927 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:52:54.905730 22927 hybrid_clock.cc:648] HybridClock initialized: now 1786207974905717 us; error 30 us; skew 500 ppm
I20260808 16:52:54.906873 22927 webserver.cc:460] Webserver started at http://127.8.24.62:43563/ using document root <none> and password file <none>
I20260808 16:52:54.907094 22927 fs_manager.cc:359] Metadata directory not provided
I20260808 16:52:54.907145 22927 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:52:54.907269 22927 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:52:54.908209 22927 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786207919384338-8288-0/minicluster-data/master-0/data/instance:
uuid: "579b5586a6814b629d59f21045b90565"
format_stamp: "Formatted at 2026-08-08 16:52:54 on dist-test-slave-6zbq"
server_key: "9a1948da9acd537da68ded683e432b51"
server_key_iv: "cad8f4a85ff46cf04e157763d364fecd"
server_key_version: "encryptionkey@0"
I20260808 16:52:54.908488 22927 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786207919384338-8288-0/minicluster-data/master-0/wal/instance:
uuid: "579b5586a6814b629d59f21045b90565"
format_stamp: "Formatted at 2026-08-08 16:52:54 on dist-test-slave-6zbq"
server_key: "9a1948da9acd537da68ded683e432b51"
server_key_iv: "cad8f4a85ff46cf04e157763d364fecd"
server_key_version: "encryptionkey@0"
I20260808 16:52:54.909669 22927 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 16:52:54.910240 22954 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:54.910382 22927 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:54.910465 22927 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786207919384338-8288-0/minicluster-data/master-0/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786207919384338-8288-0/minicluster-data/master-0/wal
uuid: "579b5586a6814b629d59f21045b90565"
format_stamp: "Formatted at 2026-08-08 16:52:54 on dist-test-slave-6zbq"
server_key: "9a1948da9acd537da68ded683e432b51"
server_key_iv: "cad8f4a85ff46cf04e157763d364fecd"
server_key_version: "encryptionkey@0"
I20260808 16:52:54.910533 22927 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786207919384338-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786207919384338-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786207919384338-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:52:54.927381 22927 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:52:54.927624 22927 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:52:54.927733 22927 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:52:54.931314 22927 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.62:40631
I20260808 16:52:54.931345 23036 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.62:40631 every 8 connection(s)
I20260808 16:52:54.931619 22927 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
I20260808 16:52:54.932169 23038 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:52:54.934484 23038 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 579b5586a6814b629d59f21045b90565: Bootstrap starting.
I20260808 16:52:54.935012 23038 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 579b5586a6814b629d59f21045b90565: Neither blocks nor log segments found. Creating new log.
I20260808 16:52:54.935315 23038 log.cc:824] T 00000000000000000000000000000000 P 579b5586a6814b629d59f21045b90565: Log is configured to *not* fsync() on all Append() calls
I20260808 16:52:54.935907 23038 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 579b5586a6814b629d59f21045b90565: No bootstrap required, opened a new log
I20260808 16:52:54.937361 23038 raft_consensus.cc:348] T 00000000000000000000000000000000 P 579b5586a6814b629d59f21045b90565 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "579b5586a6814b629d59f21045b90565" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 40631 } }
I20260808 16:52:54.937482 23038 raft_consensus.cc:374] T 00000000000000000000000000000000 P 579b5586a6814b629d59f21045b90565 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:52:54.937513 23038 raft_consensus.cc:729] T 00000000000000000000000000000000 P 579b5586a6814b629d59f21045b90565 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 579b5586a6814b629d59f21045b90565, State: Initialized, Role: FOLLOWER
I20260808 16:52:54.937641 23038 consensus_queue.cc:260] T 00000000000000000000000000000000 P 579b5586a6814b629d59f21045b90565 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "579b5586a6814b629d59f21045b90565" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 40631 } }
I20260808 16:52:54.937707 23038 raft_consensus.cc:388] T 00000000000000000000000000000000 P 579b5586a6814b629d59f21045b90565 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 16:52:54.937729 23038 raft_consensus.cc:482] T 00000000000000000000000000000000 P 579b5586a6814b629d59f21045b90565 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 16:52:54.937773 23038 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 579b5586a6814b629d59f21045b90565 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:52:54.938222 23038 raft_consensus.cc:504] T 00000000000000000000000000000000 P 579b5586a6814b629d59f21045b90565 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "579b5586a6814b629d59f21045b90565" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 40631 } }
I20260808 16:52:54.938306 23038 leader_election.cc:302] T 00000000000000000000000000000000 P 579b5586a6814b629d59f21045b90565 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 579b5586a6814b629d59f21045b90565; no voters:
I20260808 16:52:54.938476 23038 leader_election.cc:288] T 00000000000000000000000000000000 P 579b5586a6814b629d59f21045b90565 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 16:52:54.938530 23044 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 579b5586a6814b629d59f21045b90565 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:52:54.938674 23044 raft_consensus.cc:686] T 00000000000000000000000000000000 P 579b5586a6814b629d59f21045b90565 [term 1 LEADER]: Becoming Leader. State: Replica: 579b5586a6814b629d59f21045b90565, State: Running, Role: LEADER
I20260808 16:52:54.938705 23038 sys_catalog.cc:564] T 00000000000000000000000000000000 P 579b5586a6814b629d59f21045b90565 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 16:52:54.938776 23044 consensus_queue.cc:237] T 00000000000000000000000000000000 P 579b5586a6814b629d59f21045b90565 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "579b5586a6814b629d59f21045b90565" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 40631 } }
I20260808 16:52:54.939159 23047 sys_catalog.cc:455] T 00000000000000000000000000000000 P 579b5586a6814b629d59f21045b90565 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 579b5586a6814b629d59f21045b90565. Latest consensus state: current_term: 1 leader_uuid: "579b5586a6814b629d59f21045b90565" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "579b5586a6814b629d59f21045b90565" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 40631 } } }
I20260808 16:52:54.939239 23047 sys_catalog.cc:458] T 00000000000000000000000000000000 P 579b5586a6814b629d59f21045b90565 [sys.catalog]: This master's current role is: LEADER
I20260808 16:52:54.939255 23045 sys_catalog.cc:455] T 00000000000000000000000000000000 P 579b5586a6814b629d59f21045b90565 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "579b5586a6814b629d59f21045b90565" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "579b5586a6814b629d59f21045b90565" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 40631 } } }
I20260808 16:52:54.939332 23045 sys_catalog.cc:458] T 00000000000000000000000000000000 P 579b5586a6814b629d59f21045b90565 [sys.catalog]: This master's current role is: LEADER
I20260808 16:52:54.939682 23058 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 16:52:54.940151 23058 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 16:52:54.941231 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 22927
I20260808 16:52:54.941315 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786207919384338-8288-0/minicluster-data/master-0/wal/instance
I20260808 16:52:54.941500 8288 external_mini_cluster.cc:1428] Setting key b03362f0b0e779578ca7c7421469017b
I20260808 16:52:54.941603 23058 catalog_manager.cc:1347] Generated new cluster ID: 8dcf84ea53264b15bc06be6fe95142c9
I20260808 16:52:54.941644 23058 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 16:52:54.945737 23081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:54.941869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47960 (local address 127.8.24.62:40631)
0808 16:52:54.942041 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:54.942049 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:54.942081 (+ 32us) server_negotiation.cc:407] Connection header received
0808 16:52:54.942176 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:54.942181 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:54.942230 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:54.942334 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:54.942550 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:54.943395 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:54.944128 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:54.944524 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:54.944632 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:54.945158 (+ 526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:54.945162 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:54.945162 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 16:52:54.945313 (+ 151us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:54.945343 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 16:52:54.945352 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:54.945407 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:54.945474 (+ 67us) server_negotiation.cc:299] Negotiation successful
0808 16:52:54.945545 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":75,"threads_started":1}
I20260808 16:52:54.975297 23058 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 16:52:54.975729 23058 catalog_manager.cc:1504] Loading token signing keys...
I20260808 16:52:54.982765 23058 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 579b5586a6814b629d59f21045b90565: Generated new TSK 0
I20260808 16:52:54.982911 23058 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 16:52:54.992955 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786207919384338-8288-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786207919384338-8288-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786207919384338-8288-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786207919384338-8288-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.1:0
--local_ip_for_outbound_sockets=127.8.24.1
--webserver_interface=127.8.24.1
--webserver_port=0
--tserver_master_addrs=127.8.24.62:40631
--builtin_ntp_servers=127.8.24.20:38231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260808 16:52:55.088220 23083 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:52:55.088397 23083 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:52:55.088418 23083 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:52:55.089507 23083 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:52:55.089546 23083 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:52:55.089569 23083 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.1
I20260808 16:52:55.090879 23083 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:38231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786207919384338-8288-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786207919384338-8288-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786207919384338-8288-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.24.1
--webserver_port=0
--tserver_master_addrs=127.8.24.62:40631
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23083
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.1
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786207919384338-8288-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:52:55.091114 23083 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:52:55.091431 23083 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:52:55.095922 23092 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:55.095928 23095 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:55.096081 23091 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:55.096021 23083 server_base.cc:1034] running on GCE node
I20260808 16:52:55.096313 23083 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:52:55.096539 23083 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:52:55.097685 23083 hybrid_clock.cc:648] HybridClock initialized: now 1786207975097666 us; error 28 us; skew 500 ppm
I20260808 16:52:55.098839 23083 webserver.cc:460] Webserver started at http://127.8.24.1:36263/ using document root <none> and password file <none>
I20260808 16:52:55.099056 23083 fs_manager.cc:359] Metadata directory not provided
I20260808 16:52:55.099103 23083 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:52:55.099228 23083 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:52:55.100171 23083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786207919384338-8288-0/minicluster-data/ts-0/data/instance:
uuid: "dd63d81bc56c4a91bdc94a7ac437033d"
format_stamp: "Formatted at 2026-08-08 16:52:55 on dist-test-slave-6zbq"
server_key: "0a2611fdc985c425851837afaa8b09e9"
server_key_iv: "44bb3e18a463a692d762dafc395a0bcb"
server_key_version: "encryptionkey@0"
I20260808 16:52:55.100499 23083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786207919384338-8288-0/minicluster-data/ts-0/wal/instance:
uuid: "dd63d81bc56c4a91bdc94a7ac437033d"
format_stamp: "Formatted at 2026-08-08 16:52:55 on dist-test-slave-6zbq"
server_key: "0a2611fdc985c425851837afaa8b09e9"
server_key_iv: "44bb3e18a463a692d762dafc395a0bcb"
server_key_version: "encryptionkey@0"
I20260808 16:52:55.101634 23083 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 16:52:55.102247 23107 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:55.102413 23083 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 16:52:55.102492 23083 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786207919384338-8288-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786207919384338-8288-0/minicluster-data/ts-0/wal
uuid: "dd63d81bc56c4a91bdc94a7ac437033d"
format_stamp: "Formatted at 2026-08-08 16:52:55 on dist-test-slave-6zbq"
server_key: "0a2611fdc985c425851837afaa8b09e9"
server_key_iv: "44bb3e18a463a692d762dafc395a0bcb"
server_key_version: "encryptionkey@0"
I20260808 16:52:55.102552 23083 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786207919384338-8288-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786207919384338-8288-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786207919384338-8288-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:52:55.117774 23083 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:52:55.118014 23083 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:52:55.118149 23083 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:52:55.118403 23083 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:52:55.118695 23083 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:52:55.118729 23083 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:55.118754 23083 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:52:55.118779 23083 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:55.124292 23083 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.1:44171
I20260808 16:52:55.124351 23290 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.1:44171 every 8 connection(s)
I20260808 16:52:55.124601 23083 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786207919384338-8288-0/minicluster-data/ts-0/data/info.pb
I20260808 16:52:55.128510 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 23083
I20260808 16:52:55.128593 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786207919384338-8288-0/minicluster-data/ts-0/wal/instance
I20260808 16:52:55.128777 8288 external_mini_cluster.cc:1428] Setting key 200c3bd7e3afee0faf321d8580a123c3
I20260808 16:52:55.129225 23081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:55.125182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:50503 (local address 127.8.24.62:40631)
0808 16:52:55.125238 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:55.125241 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:55.125466 (+ 225us) server_negotiation.cc:407] Connection header received
0808 16:52:55.125880 (+ 414us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:55.125881 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:55.125891 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:55.125941 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:55.126572 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:55.127045 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:55.127909 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:55.128084 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:55.128128 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:55.128852 (+ 724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:55.128854 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:55.128854 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 16:52:55.128860 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:55.128875 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 16:52:55.128880 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:55.128935 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:55.129109 (+ 174us) server_negotiation.cc:299] Negotiation successful
0808 16:52:55.129135 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 16:52:55.129369 23295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:55.125178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:40631 (local address 127.8.24.1:50503)
0808 16:52:55.125353 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:55.125389 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:55.125808 (+ 419us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:55.125989 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:55.125993 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:55.126000 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:55.126473 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:55.126478 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:55.127117 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:55.127118 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:55.127851 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:55.127852 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:55.128136 (+ 284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:55.128136 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:55.128235 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:55.128714 (+ 479us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 16:52:55.128714 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:55.128740 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 16:52:55.128761 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:52:55.129027 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:52:55.129034 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:52:55.129037 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 16:52:55.129103 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 16:52:55.129192 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":68,"threads_started":1}
I20260808 16:52:55.129742 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786207919384338-8288-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786207919384338-8288-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786207919384338-8288-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786207919384338-8288-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.2:0
--local_ip_for_outbound_sockets=127.8.24.2
--webserver_interface=127.8.24.2
--webserver_port=0
--tserver_master_addrs=127.8.24.62:40631
--builtin_ntp_servers=127.8.24.20:38231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260808 16:52:55.129778 23291 heartbeater.cc:347] Connected to a master server at 127.8.24.62:40631
I20260808 16:52:55.129859 23291 heartbeater.cc:464] Registering TS with master...
I20260808 16:52:55.130019 23291 heartbeater.cc:511] Master 127.8.24.62:40631 requested a full tablet report, sending...
I20260808 16:52:55.130513 22981 ts_manager.cc:194] Registered new tserver with Master: dd63d81bc56c4a91bdc94a7ac437033d (127.8.24.1:44171)
I20260808 16:52:55.131265 22981 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.24.1:50503
W20260808 16:52:55.207847 23297 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:52:55.208017 23297 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:52:55.208036 23297 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:52:55.209077 23297 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:52:55.209108 23297 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:52:55.209131 23297 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.2
I20260808 16:52:55.210449 23297 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:38231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786207919384338-8288-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786207919384338-8288-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786207919384338-8288-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.24.2
--webserver_port=0
--tserver_master_addrs=127.8.24.62:40631
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23297
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.2
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786207919384338-8288-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:52:55.210614 23297 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:52:55.210865 23297 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 16:52:55.215236 23297 server_base.cc:1034] running on GCE node
W20260808 16:52:55.215200 23304 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:55.215193 23303 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:55.215211 23309 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:55.215584 23297 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:52:55.215802 23297 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:52:55.216958 23297 hybrid_clock.cc:648] HybridClock initialized: now 1786207975216944 us; error 23 us; skew 500 ppm
I20260808 16:52:55.218097 23297 webserver.cc:460] Webserver started at http://127.8.24.2:42899/ using document root <none> and password file <none>
I20260808 16:52:55.218266 23297 fs_manager.cc:359] Metadata directory not provided
I20260808 16:52:55.218307 23297 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:52:55.218431 23297 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:52:55.219337 23297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786207919384338-8288-0/minicluster-data/ts-1/data/instance:
uuid: "42bbd9cefec24991874e5df8514089cf"
format_stamp: "Formatted at 2026-08-08 16:52:55 on dist-test-slave-6zbq"
server_key: "ef6775474cb7dfdc258de1e99b2ad1d2"
server_key_iv: "a9743b51d6bc253ba8fa9627cfd6ccae"
server_key_version: "encryptionkey@0"
I20260808 16:52:55.219606 23297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786207919384338-8288-0/minicluster-data/ts-1/wal/instance:
uuid: "42bbd9cefec24991874e5df8514089cf"
format_stamp: "Formatted at 2026-08-08 16:52:55 on dist-test-slave-6zbq"
server_key: "ef6775474cb7dfdc258de1e99b2ad1d2"
server_key_iv: "a9743b51d6bc253ba8fa9627cfd6ccae"
server_key_version: "encryptionkey@0"
I20260808 16:52:55.220655 23297 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 16:52:55.221238 23316 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:55.221364 23297 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 16:52:55.221431 23297 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786207919384338-8288-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786207919384338-8288-0/minicluster-data/ts-1/wal
uuid: "42bbd9cefec24991874e5df8514089cf"
format_stamp: "Formatted at 2026-08-08 16:52:55 on dist-test-slave-6zbq"
server_key: "ef6775474cb7dfdc258de1e99b2ad1d2"
server_key_iv: "a9743b51d6bc253ba8fa9627cfd6ccae"
server_key_version: "encryptionkey@0"
I20260808 16:52:55.221478 23297 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786207919384338-8288-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786207919384338-8288-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786207919384338-8288-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:52:55.242941 23297 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:52:55.243136 23297 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:52:55.243222 23297 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:52:55.243367 23297 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:52:55.243593 23297 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:52:55.243623 23297 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:55.243645 23297 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:52:55.243661 23297 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:55.248698 23297 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.2:39203
I20260808 16:52:55.248756 23494 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.2:39203 every 8 connection(s)
I20260808 16:52:55.249007 23297 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786207919384338-8288-0/minicluster-data/ts-1/data/info.pb
I20260808 16:52:55.253413 23081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:55.249581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:35479 (local address 127.8.24.62:40631)
0808 16:52:55.249624 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:55.249626 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:55.249835 (+ 209us) server_negotiation.cc:407] Connection header received
0808 16:52:55.250268 (+ 433us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:55.250269 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:55.250284 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:55.250329 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:55.250931 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:55.251404 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:55.252311 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:55.252497 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:55.252551 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:55.253165 (+ 614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:55.253167 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:55.253167 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 16:52:55.253172 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:55.253188 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 16:52:55.253192 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:55.253236 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:55.253310 (+ 74us) server_negotiation.cc:299] Negotiation successful
0808 16:52:55.253337 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 16:52:55.253569 23499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:55.249567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:40631 (local address 127.8.24.2:35479)
0808 16:52:55.249734 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:55.249761 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:55.250174 (+ 413us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:55.250419 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:55.250423 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:55.250432 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:55.250850 (+ 418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:55.250853 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:55.251484 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:55.251485 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:55.252228 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:55.252230 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:55.252567 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:55.252568 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:55.252687 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:55.253050 (+ 363us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 16:52:55.253050 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:55.253074 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 16:52:55.253094 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:52:55.253241 (+ 147us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:52:55.253245 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:52:55.253248 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 16:52:55.253306 (+ 58us) client_negotiation.cc:240] Negotiation successful
0808 16:52:55.253397 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":76,"threads_started":1}
I20260808 16:52:55.253978 23495 heartbeater.cc:347] Connected to a master server at 127.8.24.62:40631
I20260808 16:52:55.254050 23495 heartbeater.cc:464] Registering TS with master...
I20260808 16:52:55.254180 23495 heartbeater.cc:511] Master 127.8.24.62:40631 requested a full tablet report, sending...
I20260808 16:52:55.254503 22981 ts_manager.cc:194] Registered new tserver with Master: 42bbd9cefec24991874e5df8514089cf (127.8.24.2:39203)
I20260808 16:52:55.254866 22981 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.24.2:35479
I20260808 16:52:55.255036 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 23297
I20260808 16:52:55.255111 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786207919384338-8288-0/minicluster-data/ts-1/wal/instance
I20260808 16:52:55.255273 8288 external_mini_cluster.cc:1428] Setting key c54d5f6d669df5f60fa7cbc3b100fbf8
I20260808 16:52:55.256044 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786207919384338-8288-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786207919384338-8288-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786207919384338-8288-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786207919384338-8288-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.3:0
--local_ip_for_outbound_sockets=127.8.24.3
--webserver_interface=127.8.24.3
--webserver_port=0
--tserver_master_addrs=127.8.24.62:40631
--builtin_ntp_servers=127.8.24.20:38231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260808 16:52:55.334018 23500 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:52:55.334205 23500 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:52:55.334225 23500 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:52:55.335335 23500 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:52:55.335372 23500 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:52:55.335395 23500 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.3
I20260808 16:52:55.336669 23500 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:38231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786207919384338-8288-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786207919384338-8288-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786207919384338-8288-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.24.3
--webserver_port=0
--tserver_master_addrs=127.8.24.62:40631
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23500
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.3
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786207919384338-8288-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:52:55.336866 23500 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:52:55.337121 23500 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:52:55.341900 23509 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:55.341924 23511 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:55.342072 23500 server_base.cc:1034] running on GCE node
W20260808 16:52:55.342175 23508 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:55.342417 23500 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:52:55.342633 23500 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:52:55.343775 23500 hybrid_clock.cc:648] HybridClock initialized: now 1786207975343762 us; error 30 us; skew 500 ppm
I20260808 16:52:55.344939 23500 webserver.cc:460] Webserver started at http://127.8.24.3:33527/ using document root <none> and password file <none>
I20260808 16:52:55.345155 23500 fs_manager.cc:359] Metadata directory not provided
I20260808 16:52:55.345198 23500 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:52:55.345324 23500 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:52:55.346306 23500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786207919384338-8288-0/minicluster-data/ts-2/data/instance:
uuid: "0ca4e0da4e23411680f66333e1c4aea5"
format_stamp: "Formatted at 2026-08-08 16:52:55 on dist-test-slave-6zbq"
server_key: "57a0cf6d7727a19b7078472846b5d530"
server_key_iv: "90574a1aa73dece3db2fc5e1eb6eac61"
server_key_version: "encryptionkey@0"
I20260808 16:52:55.346671 23500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786207919384338-8288-0/minicluster-data/ts-2/wal/instance:
uuid: "0ca4e0da4e23411680f66333e1c4aea5"
format_stamp: "Formatted at 2026-08-08 16:52:55 on dist-test-slave-6zbq"
server_key: "57a0cf6d7727a19b7078472846b5d530"
server_key_iv: "90574a1aa73dece3db2fc5e1eb6eac61"
server_key_version: "encryptionkey@0"
I20260808 16:52:55.347832 23500 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 16:52:55.348507 23520 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:55.348639 23500 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:55.348724 23500 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786207919384338-8288-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786207919384338-8288-0/minicluster-data/ts-2/wal
uuid: "0ca4e0da4e23411680f66333e1c4aea5"
format_stamp: "Formatted at 2026-08-08 16:52:55 on dist-test-slave-6zbq"
server_key: "57a0cf6d7727a19b7078472846b5d530"
server_key_iv: "90574a1aa73dece3db2fc5e1eb6eac61"
server_key_version: "encryptionkey@0"
I20260808 16:52:55.348790 23500 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786207919384338-8288-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786207919384338-8288-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786207919384338-8288-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:52:55.365072 23500 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:52:55.365309 23500 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:52:55.365424 23500 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:52:55.365609 23500 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:52:55.365890 23500 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:52:55.365923 23500 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:55.365949 23500 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:52:55.365967 23500 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:55.373090 23500 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.3:38099
I20260808 16:52:55.373124 23721 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.3:38099 every 8 connection(s)
I20260808 16:52:55.373401 23500 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786207919384338-8288-0/minicluster-data/ts-2/data/info.pb
I20260808 16:52:55.377830 23081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:55.373993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:44127 (local address 127.8.24.62:40631)
0808 16:52:55.374032 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:55.374034 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:55.374257 (+ 223us) server_negotiation.cc:407] Connection header received
0808 16:52:55.374724 (+ 467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:55.374726 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:55.374739 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:55.374788 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:55.375442 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:55.375924 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:55.376793 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:55.376968 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:55.377010 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:55.377636 (+ 626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:55.377637 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:55.377637 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 16:52:55.377642 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:55.377656 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 16:52:55.377660 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:55.377698 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:55.377747 (+ 49us) server_negotiation.cc:299] Negotiation successful
0808 16:52:55.377773 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 16:52:55.377991 23726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:55.373976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:40631 (local address 127.8.24.3:44127)
0808 16:52:55.374141 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:55.374170 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:55.374644 (+ 474us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:55.374837 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:55.374840 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:55.374848 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:55.375357 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:55.375362 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:55.375986 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:55.375987 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:55.376734 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:55.376736 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:55.377018 (+ 282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:55.377019 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:55.377135 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:55.377522 (+ 387us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 16:52:55.377522 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:55.377547 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 16:52:55.377567 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:52:55.377698 (+ 131us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:52:55.377700 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:52:55.377702 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 16:52:55.377741 (+ 39us) client_negotiation.cc:240] Negotiation successful
0808 16:52:55.377812 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":74,"threads_started":1}
I20260808 16:52:55.378429 23722 heartbeater.cc:347] Connected to a master server at 127.8.24.62:40631
I20260808 16:52:55.378511 23722 heartbeater.cc:464] Registering TS with master...
I20260808 16:52:55.378675 23722 heartbeater.cc:511] Master 127.8.24.62:40631 requested a full tablet report, sending...
I20260808 16:52:55.378966 22981 ts_manager.cc:194] Registered new tserver with Master: 0ca4e0da4e23411680f66333e1c4aea5 (127.8.24.3:38099)
I20260808 16:52:55.379282 22981 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.24.3:44127
I20260808 16:52:55.380595 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 23500
I20260808 16:52:55.380678 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786207919384338-8288-0/minicluster-data/ts-2/wal/instance
I20260808 16:52:55.380812 8288 external_mini_cluster.cc:1428] Setting key 7d8ae5475d0d8bb15a526d026c9fff1a
I20260808 16:52:55.381676 8288 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 16:52:55.383351 23081 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:52:55.382912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47964 (local address 127.8.24.62:40631)
0808 16:52:55.382965 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:55.382967 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:55.383050 (+ 83us) server_negotiation.cc:407] Connection header received
0808 16:52:55.383103 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:55.383104 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:55.383113 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:55.383157 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:55.383295 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47964: BlockingRecv error: recv got EOF from 127.0.0.1:47964 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 16:52:55.383378 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 23083
I20260808 16:52:55.423308 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 23297
I20260808 16:52:55.431975 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 23500
I20260808 16:52:55.438087 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 22927
2026-08-08T16:52:55Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (645 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-08T16:52:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T16:52:55Z Disabled control of system clock
I20260808 16:52:55.463194 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786207919384338-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786207919384338-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786207919384338-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:35731
--webserver_interface=127.8.24.62
--webserver_port=0
--builtin_ntp_servers=127.8.24.20:36987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:35731
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 16:52:55.541409 23739 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:52:55.541589 23739 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:52:55.541608 23739 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:52:55.542671 23739 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 16:52:55.542703 23739 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:52:55.542716 23739 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:52:55.542726 23739 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 16:52:55.542739 23739 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 16:52:55.543993 23739 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:36987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786207919384338-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786207919384338-8288-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:35731
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:35731
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.24.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23739
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786207919384338-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:52:55.544178 23739 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:52:55.544404 23739 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:52:55.548659 23749 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:55.548708 23752 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:55.548668 23748 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:55.548801 23739 server_base.cc:1034] running on GCE node
I20260808 16:52:55.549057 23739 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:52:55.549304 23739 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:52:55.550387 23739 hybrid_clock.cc:648] HybridClock initialized: now 1786207975550370 us; error 35 us; skew 500 ppm
I20260808 16:52:55.551563 23739 webserver.cc:460] Webserver started at http://127.8.24.62:37239/ using document root <none> and password file <none>
I20260808 16:52:55.551779 23739 fs_manager.cc:359] Metadata directory not provided
I20260808 16:52:55.551822 23739 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:52:55.551944 23739 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:52:55.552896 23739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786207919384338-8288-0/minicluster-data/master-0/data/instance:
uuid: "ebc25bb8821d4815b356f77723d5e3f2"
format_stamp: "Formatted at 2026-08-08 16:52:55 on dist-test-slave-6zbq"
server_key: "049a1737d6b4bf3185303531b760cd01"
server_key_iv: "f7b070c05fd82143d0b4191fce34df79"
server_key_version: "encryptionkey@0"
I20260808 16:52:55.553205 23739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786207919384338-8288-0/minicluster-data/master-0/wal/instance:
uuid: "ebc25bb8821d4815b356f77723d5e3f2"
format_stamp: "Formatted at 2026-08-08 16:52:55 on dist-test-slave-6zbq"
server_key: "049a1737d6b4bf3185303531b760cd01"
server_key_iv: "f7b070c05fd82143d0b4191fce34df79"
server_key_version: "encryptionkey@0"
I20260808 16:52:55.554409 23739 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 16:52:55.555068 23765 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:55.555202 23739 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 16:52:55.555284 23739 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786207919384338-8288-0/minicluster-data/master-0/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786207919384338-8288-0/minicluster-data/master-0/wal
uuid: "ebc25bb8821d4815b356f77723d5e3f2"
format_stamp: "Formatted at 2026-08-08 16:52:55 on dist-test-slave-6zbq"
server_key: "049a1737d6b4bf3185303531b760cd01"
server_key_iv: "f7b070c05fd82143d0b4191fce34df79"
server_key_version: "encryptionkey@0"
I20260808 16:52:55.555354 23739 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786207919384338-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786207919384338-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786207919384338-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:52:55.558660 23739 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:52:55.558905 23739 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:52:55.559027 23739 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:52:55.562523 23739 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.62:35731
I20260808 16:52:55.562552 23846 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.62:35731 every 8 connection(s)
I20260808 16:52:55.562839 23739 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
I20260808 16:52:55.563421 23847 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:52:55.565896 23847 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ebc25bb8821d4815b356f77723d5e3f2: Bootstrap starting.
I20260808 16:52:55.566428 23847 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ebc25bb8821d4815b356f77723d5e3f2: Neither blocks nor log segments found. Creating new log.
I20260808 16:52:55.566700 23847 log.cc:824] T 00000000000000000000000000000000 P ebc25bb8821d4815b356f77723d5e3f2: Log is configured to *not* fsync() on all Append() calls
I20260808 16:52:55.566939 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 23739
I20260808 16:52:55.567032 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786207919384338-8288-0/minicluster-data/master-0/wal/instance
I20260808 16:52:55.567252 8288 external_mini_cluster.cc:1428] Setting key 2eb03d1dfc9e951baf1a1f1b9d4ae72b
I20260808 16:52:55.567445 23847 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ebc25bb8821d4815b356f77723d5e3f2: No bootstrap required, opened a new log
I20260808 16:52:55.569419 23847 raft_consensus.cc:348] T 00000000000000000000000000000000 P ebc25bb8821d4815b356f77723d5e3f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebc25bb8821d4815b356f77723d5e3f2" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35731 } }
I20260808 16:52:55.569573 23847 raft_consensus.cc:374] T 00000000000000000000000000000000 P ebc25bb8821d4815b356f77723d5e3f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:52:55.569623 23847 raft_consensus.cc:729] T 00000000000000000000000000000000 P ebc25bb8821d4815b356f77723d5e3f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ebc25bb8821d4815b356f77723d5e3f2, State: Initialized, Role: FOLLOWER
I20260808 16:52:55.569772 23854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:55.567888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48526 (local address 127.8.24.62:35731)
0808 16:52:55.568053 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:55.568060 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:55.568083 (+ 23us) server_negotiation.cc:407] Connection header received
0808 16:52:55.568618 (+ 535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:55.568624 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:55.568659 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:55.568772 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:55.569360 (+ 588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:55.569363 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:55.569363 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 16:52:55.569367 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:55.569403 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 16:52:55.569412 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:55.569462 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:55.569542 (+ 80us) server_negotiation.cc:299] Negotiation successful
0808 16:52:55.569588 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":57,"threads_started":1}
I20260808 16:52:55.569752 23847 consensus_queue.cc:260] T 00000000000000000000000000000000 P ebc25bb8821d4815b356f77723d5e3f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebc25bb8821d4815b356f77723d5e3f2" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35731 } }
I20260808 16:52:55.569833 23847 raft_consensus.cc:388] T 00000000000000000000000000000000 P ebc25bb8821d4815b356f77723d5e3f2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 16:52:55.569865 23847 raft_consensus.cc:482] T 00000000000000000000000000000000 P ebc25bb8821d4815b356f77723d5e3f2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 16:52:55.569904 23847 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ebc25bb8821d4815b356f77723d5e3f2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:52:55.570760 23847 raft_consensus.cc:504] T 00000000000000000000000000000000 P ebc25bb8821d4815b356f77723d5e3f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebc25bb8821d4815b356f77723d5e3f2" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35731 } }
I20260808 16:52:55.570844 23847 leader_election.cc:302] T 00000000000000000000000000000000 P ebc25bb8821d4815b356f77723d5e3f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ebc25bb8821d4815b356f77723d5e3f2; no voters:
I20260808 16:52:55.570955 23847 leader_election.cc:288] T 00000000000000000000000000000000 P ebc25bb8821d4815b356f77723d5e3f2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 16:52:55.571002 23855 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ebc25bb8821d4815b356f77723d5e3f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:52:55.571143 23855 raft_consensus.cc:686] T 00000000000000000000000000000000 P ebc25bb8821d4815b356f77723d5e3f2 [term 1 LEADER]: Becoming Leader. State: Replica: ebc25bb8821d4815b356f77723d5e3f2, State: Running, Role: LEADER
I20260808 16:52:55.571177 23847 sys_catalog.cc:564] T 00000000000000000000000000000000 P ebc25bb8821d4815b356f77723d5e3f2 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 16:52:55.571239 23855 consensus_queue.cc:237] T 00000000000000000000000000000000 P ebc25bb8821d4815b356f77723d5e3f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebc25bb8821d4815b356f77723d5e3f2" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35731 } }
I20260808 16:52:55.571576 23857 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebc25bb8821d4815b356f77723d5e3f2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ebc25bb8821d4815b356f77723d5e3f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebc25bb8821d4815b356f77723d5e3f2" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35731 } } }
I20260808 16:52:55.571590 23860 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebc25bb8821d4815b356f77723d5e3f2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ebc25bb8821d4815b356f77723d5e3f2. Latest consensus state: current_term: 1 leader_uuid: "ebc25bb8821d4815b356f77723d5e3f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebc25bb8821d4815b356f77723d5e3f2" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35731 } } }
I20260808 16:52:55.571758 23857 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebc25bb8821d4815b356f77723d5e3f2 [sys.catalog]: This master's current role is: LEADER
I20260808 16:52:55.571776 23860 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebc25bb8821d4815b356f77723d5e3f2 [sys.catalog]: This master's current role is: LEADER
I20260808 16:52:55.572113 23873 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 16:52:55.572518 23873 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 16:52:55.573770 23873 catalog_manager.cc:1347] Generated new cluster ID: 186e817ca1d94b179b70cc4133ea2feb
I20260808 16:52:55.573825 23873 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 16:52:55.587766 23873 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 16:52:55.587908 23873 catalog_manager.cc:1504] Loading token signing keys...
I20260808 16:52:55.592274 23873 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ebc25bb8821d4815b356f77723d5e3f2: Generated new TSK 0
I20260808 16:52:55.592453 23873 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 16:52:55.596789 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786207919384338-8288-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786207919384338-8288-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786207919384338-8288-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786207919384338-8288-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.1:0
--local_ip_for_outbound_sockets=127.8.24.1
--webserver_interface=127.8.24.1
--webserver_port=0
--tserver_master_addrs=127.8.24.62:35731
--builtin_ntp_servers=127.8.24.20:36987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 16:52:55.676267 23888 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:52:55.676450 23888 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:52:55.676479 23888 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:52:55.677572 23888 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:52:55.677608 23888 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:52:55.677642 23888 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.1
I20260808 16:52:55.678997 23888 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:36987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786207919384338-8288-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786207919384338-8288-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786207919384338-8288-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.24.1
--webserver_port=0
--tserver_master_addrs=127.8.24.62:35731
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23888
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.1
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786207919384338-8288-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:52:55.679208 23888 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:52:55.679462 23888 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:52:55.683945 23893 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:55.683946 23896 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:55.683957 23900 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:55.684217 23888 server_base.cc:1034] running on GCE node
I20260808 16:52:55.684397 23888 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:52:55.684631 23888 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:52:55.685779 23888 hybrid_clock.cc:648] HybridClock initialized: now 1786207975685765 us; error 31 us; skew 500 ppm
I20260808 16:52:55.686975 23888 webserver.cc:460] Webserver started at http://127.8.24.1:35127/ using document root <none> and password file <none>
I20260808 16:52:55.687192 23888 fs_manager.cc:359] Metadata directory not provided
I20260808 16:52:55.687244 23888 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:52:55.687363 23888 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:52:55.688313 23888 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786207919384338-8288-0/minicluster-data/ts-0/data/instance:
uuid: "fc518cf98acb47b59b978f4c49051c3a"
format_stamp: "Formatted at 2026-08-08 16:52:55 on dist-test-slave-6zbq"
server_key: "6c9138e7fbc21c4680156ec6ead3489b"
server_key_iv: "6c05d4e0b437706f7b3e2bb0d8d2dce8"
server_key_version: "encryptionkey@0"
I20260808 16:52:55.688624 23888 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786207919384338-8288-0/minicluster-data/ts-0/wal/instance:
uuid: "fc518cf98acb47b59b978f4c49051c3a"
format_stamp: "Formatted at 2026-08-08 16:52:55 on dist-test-slave-6zbq"
server_key: "6c9138e7fbc21c4680156ec6ead3489b"
server_key_iv: "6c05d4e0b437706f7b3e2bb0d8d2dce8"
server_key_version: "encryptionkey@0"
I20260808 16:52:55.689793 23888 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 16:52:55.690480 23911 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:55.690639 23888 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 16:52:55.690735 23888 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786207919384338-8288-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786207919384338-8288-0/minicluster-data/ts-0/wal
uuid: "fc518cf98acb47b59b978f4c49051c3a"
format_stamp: "Formatted at 2026-08-08 16:52:55 on dist-test-slave-6zbq"
server_key: "6c9138e7fbc21c4680156ec6ead3489b"
server_key_iv: "6c05d4e0b437706f7b3e2bb0d8d2dce8"
server_key_version: "encryptionkey@0"
I20260808 16:52:55.690804 23888 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786207919384338-8288-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786207919384338-8288-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786207919384338-8288-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:52:55.695286 23888 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:52:55.695547 23888 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:52:55.695667 23888 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:52:55.695881 23888 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:52:55.696231 23888 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:52:55.696282 23888 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:55.696316 23888 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:52:55.696341 23888 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:55.701572 23888 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.1:40571
I20260808 16:52:55.701655 24086 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.1:40571 every 8 connection(s)
I20260808 16:52:55.701900 23888 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786207919384338-8288-0/minicluster-data/ts-0/data/info.pb
I20260808 16:52:55.704557 23854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:55.702650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:56871 (local address 127.8.24.62:35731)
0808 16:52:55.702700 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:55.702702 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:55.702923 (+ 221us) server_negotiation.cc:407] Connection header received
0808 16:52:55.703333 (+ 410us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:55.703334 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:55.703338 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:55.703388 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:55.704376 (+ 988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:55.704377 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:55.704378 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 16:52:55.704381 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:55.704395 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 16:52:55.704399 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:55.704441 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:55.704496 (+ 55us) server_negotiation.cc:299] Negotiation successful
0808 16:52:55.704507 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 16:52:55.704730 24089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:55.702636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:35731 (local address 127.8.24.1:56871)
0808 16:52:55.702819 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:55.702854 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:55.703260 (+ 406us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:55.703444 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:55.703447 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:55.703455 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:55.704244 (+ 789us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 16:52:55.704245 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:55.704272 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 16:52:55.704299 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:52:55.704447 (+ 148us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:52:55.704449 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:52:55.704451 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 16:52:55.704497 (+ 46us) client_negotiation.cc:240] Negotiation successful
0808 16:52:55.704541 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":72,"threads_started":1}
I20260808 16:52:55.705224 24087 heartbeater.cc:347] Connected to a master server at 127.8.24.62:35731
I20260808 16:52:55.705303 24087 heartbeater.cc:464] Registering TS with master...
I20260808 16:52:55.705444 24087 heartbeater.cc:511] Master 127.8.24.62:35731 requested a full tablet report, sending...
I20260808 16:52:55.705880 23791 ts_manager.cc:194] Registered new tserver with Master: fc518cf98acb47b59b978f4c49051c3a (127.8.24.1:40571)
I20260808 16:52:55.711863 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 23888
I20260808 16:52:55.711935 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786207919384338-8288-0/minicluster-data/ts-0/wal/instance
I20260808 16:52:55.712101 8288 external_mini_cluster.cc:1428] Setting key 46bb12cdd1e8366caa3f44ecc0f962b1
I20260808 16:52:55.712893 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786207919384338-8288-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786207919384338-8288-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786207919384338-8288-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786207919384338-8288-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.2:0
--local_ip_for_outbound_sockets=127.8.24.2
--webserver_interface=127.8.24.2
--webserver_port=0
--tserver_master_addrs=127.8.24.62:35731
--builtin_ntp_servers=127.8.24.20:36987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 16:52:55.791654 24090 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:52:55.791839 24090 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:52:55.791867 24090 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:52:55.793090 24090 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:52:55.793138 24090 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:52:55.793171 24090 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.2
I20260808 16:52:55.794541 24090 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:36987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786207919384338-8288-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786207919384338-8288-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786207919384338-8288-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.24.2
--webserver_port=0
--tserver_master_addrs=127.8.24.62:35731
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24090
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.2
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786207919384338-8288-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:52:55.794751 24090 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:52:55.795002 24090 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 16:52:55.799889 24090 server_base.cc:1034] running on GCE node
W20260808 16:52:55.799835 24097 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:55.799841 24096 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:55.799849 24100 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:55.800199 24090 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:52:55.800406 24090 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:52:55.801540 24090 hybrid_clock.cc:648] HybridClock initialized: now 1786207975801529 us; error 28 us; skew 500 ppm
I20260808 16:52:55.802702 24090 webserver.cc:460] Webserver started at http://127.8.24.2:43389/ using document root <none> and password file <none>
I20260808 16:52:55.802899 24090 fs_manager.cc:359] Metadata directory not provided
I20260808 16:52:55.802942 24090 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:52:55.803054 24090 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:52:55.804003 24090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786207919384338-8288-0/minicluster-data/ts-1/data/instance:
uuid: "23b9a31fe8904764b41efee83d11a5b7"
format_stamp: "Formatted at 2026-08-08 16:52:55 on dist-test-slave-6zbq"
server_key: "b55003a7a7c81b71b0555a79fb917054"
server_key_iv: "f2b13d39eb44c14530f1ee1be01edd41"
server_key_version: "encryptionkey@0"
I20260808 16:52:55.804311 24090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786207919384338-8288-0/minicluster-data/ts-1/wal/instance:
uuid: "23b9a31fe8904764b41efee83d11a5b7"
format_stamp: "Formatted at 2026-08-08 16:52:55 on dist-test-slave-6zbq"
server_key: "b55003a7a7c81b71b0555a79fb917054"
server_key_iv: "f2b13d39eb44c14530f1ee1be01edd41"
server_key_version: "encryptionkey@0"
I20260808 16:52:55.805456 24090 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 16:52:55.806087 24112 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:55.806226 24090 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 16:52:55.806305 24090 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786207919384338-8288-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786207919384338-8288-0/minicluster-data/ts-1/wal
uuid: "23b9a31fe8904764b41efee83d11a5b7"
format_stamp: "Formatted at 2026-08-08 16:52:55 on dist-test-slave-6zbq"
server_key: "b55003a7a7c81b71b0555a79fb917054"
server_key_iv: "f2b13d39eb44c14530f1ee1be01edd41"
server_key_version: "encryptionkey@0"
I20260808 16:52:55.806391 24090 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786207919384338-8288-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786207919384338-8288-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786207919384338-8288-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:52:55.809482 24090 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:52:55.809731 24090 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:52:55.809851 24090 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:52:55.810032 24090 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:52:55.810310 24090 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:52:55.810359 24090 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:55.810387 24090 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:52:55.810413 24090 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:55.815761 24090 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.2:43283
I20260808 16:52:55.815820 24292 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.2:43283 every 8 connection(s)
I20260808 16:52:55.816108 24090 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786207919384338-8288-0/minicluster-data/ts-1/data/info.pb
I20260808 16:52:55.817420 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 24090
I20260808 16:52:55.817515 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786207919384338-8288-0/minicluster-data/ts-1/wal/instance
I20260808 16:52:55.817689 8288 external_mini_cluster.cc:1428] Setting key 9f7a298d8de2315b9a7f7053d1bb5a7e
I20260808 16:52:55.818650 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786207919384338-8288-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786207919384338-8288-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786207919384338-8288-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786207919384338-8288-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.3:0
--local_ip_for_outbound_sockets=127.8.24.3
--webserver_interface=127.8.24.3
--webserver_port=0
--tserver_master_addrs=127.8.24.62:35731
--builtin_ntp_servers=127.8.24.20:36987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260808 16:52:55.819047 23854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:55.816757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:60821 (local address 127.8.24.62:35731)
0808 16:52:55.816798 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:55.816800 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:55.817038 (+ 238us) server_negotiation.cc:407] Connection header received
0808 16:52:55.817528 (+ 490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:55.817529 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:55.817536 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:55.817588 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:55.818811 (+ 1223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:55.818812 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:55.818812 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 16:52:55.818817 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:55.818835 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 16:52:55.818840 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:55.818867 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:55.818963 (+ 96us) server_negotiation.cc:299] Negotiation successful
0808 16:52:55.818978 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 16:52:55.819232 24298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:55.816747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:35731 (local address 127.8.24.2:60821)
0808 16:52:55.816919 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:55.816951 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:55.817424 (+ 473us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:55.817658 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:55.817662 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:55.817673 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:55.818654 (+ 981us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 16:52:55.818655 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:55.818686 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 16:52:55.818715 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:52:55.818897 (+ 182us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:52:55.818901 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:52:55.818903 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 16:52:55.818961 (+ 58us) client_negotiation.cc:240] Negotiation successful
0808 16:52:55.819022 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":76,"threads_started":1}
I20260808 16:52:55.819628 24296 heartbeater.cc:347] Connected to a master server at 127.8.24.62:35731
I20260808 16:52:55.819721 24296 heartbeater.cc:464] Registering TS with master...
I20260808 16:52:55.819867 24296 heartbeater.cc:511] Master 127.8.24.62:35731 requested a full tablet report, sending...
I20260808 16:52:55.820225 23791 ts_manager.cc:194] Registered new tserver with Master: 23b9a31fe8904764b41efee83d11a5b7 (127.8.24.2:43283)
W20260808 16:52:55.897775 24299 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:52:55.897949 24299 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:52:55.897969 24299 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:52:55.899075 24299 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:52:55.899112 24299 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:52:55.899138 24299 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.3
I20260808 16:52:55.900431 24299 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:36987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786207919384338-8288-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786207919384338-8288-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786207919384338-8288-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.24.3
--webserver_port=0
--tserver_master_addrs=127.8.24.62:35731
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24299
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.3
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786207919384338-8288-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:52:55.900635 24299 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:52:55.900879 24299 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 16:52:55.905360 24299 server_base.cc:1034] running on GCE node
W20260808 16:52:55.905344 24315 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:55.905344 24309 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:55.905318 24311 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:55.905704 24299 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:52:55.905913 24299 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:52:55.907050 24299 hybrid_clock.cc:648] HybridClock initialized: now 1786207975907035 us; error 27 us; skew 500 ppm
I20260808 16:52:55.908109 24299 webserver.cc:460] Webserver started at http://127.8.24.3:38363/ using document root <none> and password file <none>
I20260808 16:52:55.908313 24299 fs_manager.cc:359] Metadata directory not provided
I20260808 16:52:55.908358 24299 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:52:55.908483 24299 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:52:55.909505 24299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786207919384338-8288-0/minicluster-data/ts-2/data/instance:
uuid: "7d1dd02cc8ee4935bbe5fe0dc7d5e832"
format_stamp: "Formatted at 2026-08-08 16:52:55 on dist-test-slave-6zbq"
server_key: "2f755522137d85d653f886efa23292e8"
server_key_iv: "d60b65af2c31694b0ada9d131cfb7990"
server_key_version: "encryptionkey@0"
I20260808 16:52:55.909871 24299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786207919384338-8288-0/minicluster-data/ts-2/wal/instance:
uuid: "7d1dd02cc8ee4935bbe5fe0dc7d5e832"
format_stamp: "Formatted at 2026-08-08 16:52:55 on dist-test-slave-6zbq"
server_key: "2f755522137d85d653f886efa23292e8"
server_key_iv: "d60b65af2c31694b0ada9d131cfb7990"
server_key_version: "encryptionkey@0"
I20260808 16:52:55.911044 24299 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 16:52:55.911629 24322 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:55.911774 24299 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 16:52:55.911847 24299 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786207919384338-8288-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786207919384338-8288-0/minicluster-data/ts-2/wal
uuid: "7d1dd02cc8ee4935bbe5fe0dc7d5e832"
format_stamp: "Formatted at 2026-08-08 16:52:55 on dist-test-slave-6zbq"
server_key: "2f755522137d85d653f886efa23292e8"
server_key_iv: "d60b65af2c31694b0ada9d131cfb7990"
server_key_version: "encryptionkey@0"
I20260808 16:52:55.911911 24299 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786207919384338-8288-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786207919384338-8288-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786207919384338-8288-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:52:55.914985 24299 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:52:55.915194 24299 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:52:55.915302 24299 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:52:55.915479 24299 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:52:55.915766 24299 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:52:55.915803 24299 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:55.915830 24299 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:52:55.915853 24299 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:55.920938 24299 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.3:43295
I20260808 16:52:55.920974 24506 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.3:43295 every 8 connection(s)
I20260808 16:52:55.921236 24299 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786207919384338-8288-0/minicluster-data/ts-2/data/info.pb
I20260808 16:52:55.923589 23854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:55.921829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:44811 (local address 127.8.24.62:35731)
0808 16:52:55.921875 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:55.921877 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:55.922052 (+ 175us) server_negotiation.cc:407] Connection header received
0808 16:52:55.922471 (+ 419us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:55.922472 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:55.922478 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:55.922516 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:55.923419 (+ 903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:55.923420 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:55.923420 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 16:52:55.923425 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:55.923439 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 16:52:55.923443 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:55.923484 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:55.923527 (+ 43us) server_negotiation.cc:299] Negotiation successful
0808 16:52:55.923539 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 16:52:55.923774 24510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:55.921809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:35731 (local address 127.8.24.3:44811)
0808 16:52:55.921954 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:55.921983 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:55.922382 (+ 399us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:55.922563 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:55.922566 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:55.922573 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:55.923300 (+ 727us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 16:52:55.923301 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:55.923325 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 16:52:55.923348 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:52:55.923484 (+ 136us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:52:55.923486 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:52:55.923488 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 16:52:55.923532 (+ 44us) client_negotiation.cc:240] Negotiation successful
0808 16:52:55.923574 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":71,"threads_started":1}
I20260808 16:52:55.923797 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 24299
I20260808 16:52:55.923864 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786207919384338-8288-0/minicluster-data/ts-2/wal/instance
I20260808 16:52:55.924013 8288 external_mini_cluster.cc:1428] Setting key 055f7f083957affc79d2acc58818b8c2
I20260808 16:52:55.924297 24508 heartbeater.cc:347] Connected to a master server at 127.8.24.62:35731
I20260808 16:52:55.924386 24508 heartbeater.cc:464] Registering TS with master...
I20260808 16:52:55.924527 24508 heartbeater.cc:511] Master 127.8.24.62:35731 requested a full tablet report, sending...
I20260808 16:52:55.924871 23791 ts_manager.cc:194] Registered new tserver with Master: 7d1dd02cc8ee4935bbe5fe0dc7d5e832 (127.8.24.3:43295)
I20260808 16:52:55.935458 8288 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 16:52:55.936980 23854 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:52:55.936634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47406 (local address 127.8.24.62:35731)
0808 16:52:55.936672 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:55.936674 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:55.936758 (+ 84us) server_negotiation.cc:407] Connection header received
0808 16:52:55.936800 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:55.936801 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:55.936804 (+ 3us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:55.936845 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:55.936941 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47406: BlockingRecv error: recv got EOF from 127.0.0.1:47406 (error 108)
Metrics: {"server-negotiator.queue_time_us":14}
I20260808 16:52:55.937058 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 23888
I20260808 16:52:55.947602 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 24090
I20260808 16:52:55.961792 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 24299
I20260808 16:52:55.967366 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 23739
2026-08-08T16:52:55Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (533 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786207919384338-8288-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 16:52:56 dist-test-slave-6zbq krb5kdc[24526](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 16:52:56 dist-test-slave-6zbq krb5kdc[24526](info): set up 2 sockets
Aug 08 16:52:56 dist-test-slave-6zbq krb5kdc[24526](info): commencing operation
krb5kdc: starting...
W20260808 16:52:57.106453 8288 mini_kdc.cc:121] Time spent starting KDC: real 1.117s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786207919384338-8288-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786207919384338-8288-0/krb5kdc/test-user.keytab.
Aug 08 16:52:57 dist-test-slave-6zbq krb5kdc[24526](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207977, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T16:52:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T16:52:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.24.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
I20260808 16:52:57.210246 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786207919384338-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786207919384338-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786207919384338-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:41097
--webserver_interface=127.8.24.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:33347
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:41097
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:52:57.289789 24550 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:52:57.289980 24550 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:52:57.290009 24550 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:52:57.291157 24550 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 16:52:57.291198 24550 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:52:57.291216 24550 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:52:57.291232 24550 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 16:52:57.291249 24550 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 16:52:57.292682 24550 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:33347
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786207919384338-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786207919384338-8288-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:41097
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:41097
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.24.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24550
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786207919384338-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:52:57.292901 24550 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:52:57.293105 24550 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:52:57.297668 24561 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:57.297689 24558 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:57.297694 24559 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:57.297755 24550 server_base.cc:1034] running on GCE node
I20260808 16:52:57.298069 24550 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:52:57.298372 24550 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:52:57.299517 24550 hybrid_clock.cc:648] HybridClock initialized: now 1786207977299504 us; error 30 us; skew 500 ppm
Aug 08 16:52:57 dist-test-slave-6zbq krb5kdc[24526](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207977, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:52:57.301882 24550 init.cc:377] Logged in from keytab as kudu/127.8.24.62@KRBTEST.COM (short username kudu)
I20260808 16:52:57.302269 24550 webserver.cc:460] Webserver started at http://127.8.24.62:44879/ using document root <none> and password file <none>
I20260808 16:52:57.302500 24550 fs_manager.cc:359] Metadata directory not provided
I20260808 16:52:57.302551 24550 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:52:57.302677 24550 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:52:57.303660 24550 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786207919384338-8288-0/minicluster-data/master-0/data/instance:
uuid: "cade7cc8ece44438802332a59875f2b5"
format_stamp: "Formatted at 2026-08-08 16:52:57 on dist-test-slave-6zbq"
server_key: "60d71406dc1a4041fed7e713d50d1444"
server_key_iv: "6149d93dcb16182a653d038799fc9d73"
server_key_version: "encryptionkey@0"
I20260808 16:52:57.303989 24550 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786207919384338-8288-0/minicluster-data/master-0/wal/instance:
uuid: "cade7cc8ece44438802332a59875f2b5"
format_stamp: "Formatted at 2026-08-08 16:52:57 on dist-test-slave-6zbq"
server_key: "60d71406dc1a4041fed7e713d50d1444"
server_key_iv: "6149d93dcb16182a653d038799fc9d73"
server_key_version: "encryptionkey@0"
I20260808 16:52:57.305199 24550 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 16:52:57.305843 24571 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:57.305991 24550 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:57.306073 24550 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786207919384338-8288-0/minicluster-data/master-0/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786207919384338-8288-0/minicluster-data/master-0/wal
uuid: "cade7cc8ece44438802332a59875f2b5"
format_stamp: "Formatted at 2026-08-08 16:52:57 on dist-test-slave-6zbq"
server_key: "60d71406dc1a4041fed7e713d50d1444"
server_key_iv: "6149d93dcb16182a653d038799fc9d73"
server_key_version: "encryptionkey@0"
I20260808 16:52:57.306134 24550 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786207919384338-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786207919384338-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786207919384338-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:52:57.332010 24550 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:52:57.333652 24550 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:52:57.333781 24550 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:52:57.337460 24550 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.62:41097
I20260808 16:52:57.337529 24648 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.62:41097 every 8 connection(s)
I20260808 16:52:57.337816 24550 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
I20260808 16:52:57.338842 24649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:52:57.341012 24649 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cade7cc8ece44438802332a59875f2b5: Bootstrap starting.
I20260808 16:52:57.341572 24649 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cade7cc8ece44438802332a59875f2b5: Neither blocks nor log segments found. Creating new log.
I20260808 16:52:57.341827 24649 log.cc:824] T 00000000000000000000000000000000 P cade7cc8ece44438802332a59875f2b5: Log is configured to *not* fsync() on all Append() calls
I20260808 16:52:57.342496 24649 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cade7cc8ece44438802332a59875f2b5: No bootstrap required, opened a new log
I20260808 16:52:57.344015 24649 raft_consensus.cc:348] T 00000000000000000000000000000000 P cade7cc8ece44438802332a59875f2b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cade7cc8ece44438802332a59875f2b5" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 41097 } }
I20260808 16:52:57.344156 24649 raft_consensus.cc:374] T 00000000000000000000000000000000 P cade7cc8ece44438802332a59875f2b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:52:57.344179 24649 raft_consensus.cc:729] T 00000000000000000000000000000000 P cade7cc8ece44438802332a59875f2b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cade7cc8ece44438802332a59875f2b5, State: Initialized, Role: FOLLOWER
I20260808 16:52:57.344303 24649 consensus_queue.cc:260] T 00000000000000000000000000000000 P cade7cc8ece44438802332a59875f2b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cade7cc8ece44438802332a59875f2b5" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 41097 } }
I20260808 16:52:57.344377 24649 raft_consensus.cc:388] T 00000000000000000000000000000000 P cade7cc8ece44438802332a59875f2b5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 16:52:57.344410 24649 raft_consensus.cc:482] T 00000000000000000000000000000000 P cade7cc8ece44438802332a59875f2b5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 16:52:57.344458 24649 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cade7cc8ece44438802332a59875f2b5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:52:57.344476 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 24550
I20260808 16:52:57.344568 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786207919384338-8288-0/minicluster-data/master-0/wal/instance
I20260808 16:52:57.344759 8288 external_mini_cluster.cc:1428] Setting key 4afd3e2cf6306a6bd4fdcd39ff273e6e
I20260808 16:52:57.345403 24649 raft_consensus.cc:504] T 00000000000000000000000000000000 P cade7cc8ece44438802332a59875f2b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cade7cc8ece44438802332a59875f2b5" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 41097 } }
I20260808 16:52:57.345515 24649 leader_election.cc:302] T 00000000000000000000000000000000 P cade7cc8ece44438802332a59875f2b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cade7cc8ece44438802332a59875f2b5; no voters:
I20260808 16:52:57.345672 24649 leader_election.cc:288] T 00000000000000000000000000000000 P cade7cc8ece44438802332a59875f2b5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 16:52:57.345708 24656 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cade7cc8ece44438802332a59875f2b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:52:57.345861 24656 raft_consensus.cc:686] T 00000000000000000000000000000000 P cade7cc8ece44438802332a59875f2b5 [term 1 LEADER]: Becoming Leader. State: Replica: cade7cc8ece44438802332a59875f2b5, State: Running, Role: LEADER
I20260808 16:52:57.345907 24649 sys_catalog.cc:564] T 00000000000000000000000000000000 P cade7cc8ece44438802332a59875f2b5 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 16:52:57.347222 24656 consensus_queue.cc:237] T 00000000000000000000000000000000 P cade7cc8ece44438802332a59875f2b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cade7cc8ece44438802332a59875f2b5" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 41097 } }
I20260808 16:52:57.347905 24658 sys_catalog.cc:455] T 00000000000000000000000000000000 P cade7cc8ece44438802332a59875f2b5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cade7cc8ece44438802332a59875f2b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cade7cc8ece44438802332a59875f2b5" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 41097 } } }
I20260808 16:52:57.347965 24658 sys_catalog.cc:458] T 00000000000000000000000000000000 P cade7cc8ece44438802332a59875f2b5 [sys.catalog]: This master's current role is: LEADER
I20260808 16:52:57.347779 24657 sys_catalog.cc:455] T 00000000000000000000000000000000 P cade7cc8ece44438802332a59875f2b5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cade7cc8ece44438802332a59875f2b5. Latest consensus state: current_term: 1 leader_uuid: "cade7cc8ece44438802332a59875f2b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cade7cc8ece44438802332a59875f2b5" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 41097 } } }
I20260808 16:52:57.348083 24657 sys_catalog.cc:458] T 00000000000000000000000000000000 P cade7cc8ece44438802332a59875f2b5 [sys.catalog]: This master's current role is: LEADER
I20260808 16:52:57.348227 24680 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 16:52:57.348703 24680 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 16:52:57.350260 24680 catalog_manager.cc:1347] Generated new cluster ID: e00b46b9293448c1af4e3bb6434889a2
I20260808 16:52:57.350312 24680 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 16:52:57 dist-test-slave-6zbq krb5kdc[24526](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207977, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260808 16:52:57.356575 24654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:57.345218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42814 (local address 127.8.24.62:41097)
0808 16:52:57.345387 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:57.345403 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:57.345439 (+ 36us) server_negotiation.cc:407] Connection header received
0808 16:52:57.345553 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:57.345560 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:57.345616 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:57.345773 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:57.346570 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:57.348070 (+ 1500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:57.349033 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:57.349634 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:57.349755 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:57.352725 (+ 2970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:57.352730 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:57.352730 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:57.352741 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:57.355024 (+ 2283us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:57.355405 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:57.355408 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:57.355408 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:57.355513 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:57.355719 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:57.355720 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:57.355720 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:57.355985 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:57.356134 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:57.356236 (+ 102us) server_negotiation.cc:299] Negotiation successful
0808 16:52:57.356363 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":78,"threads_started":1}
I20260808 16:52:57.364933 24680 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 16:52:57.365439 24680 catalog_manager.cc:1504] Loading token signing keys...
I20260808 16:52:57.368490 24680 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cade7cc8ece44438802332a59875f2b5: Generated new TSK 0
I20260808 16:52:57.368635 24680 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.24.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.1.keytab.
Entry for principal kudu/127.8.24.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.1.keytab.
I20260808 16:52:57.393124 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786207919384338-8288-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786207919384338-8288-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786207919384338-8288-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786207919384338-8288-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.1:0
--local_ip_for_outbound_sockets=127.8.24.1
--webserver_interface=127.8.24.1
--webserver_port=0
--tserver_master_addrs=127.8.24.62:41097
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.1.keytab
--principal=kudu/127.8.24.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:33347
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:52:57.470983 24687 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:52:57.471163 24687 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:52:57.471182 24687 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:52:57.472270 24687 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:52:57.472308 24687 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:52:57.472337 24687 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.1
I20260808 16:52:57.473716 24687 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:33347
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786207919384338-8288-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786207919384338-8288-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.1.keytab
--principal=kudu/127.8.24.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786207919384338-8288-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.24.1
--webserver_port=0
--tserver_master_addrs=127.8.24.62:41097
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24687
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.1
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786207919384338-8288-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:52:57.473922 24687 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:52:57.474113 24687 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:52:57.478736 24692 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:57.478852 24687 server_base.cc:1034] running on GCE node
W20260808 16:52:57.478753 24696 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:57.478742 24693 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:57.479161 24687 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:52:57.479382 24687 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:52:57.480513 24687 hybrid_clock.cc:648] HybridClock initialized: now 1786207977480501 us; error 29 us; skew 500 ppm
Aug 08 16:52:57 dist-test-slave-6zbq krb5kdc[24526](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207977, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:52:57.482916 24687 init.cc:377] Logged in from keytab as kudu/127.8.24.1@KRBTEST.COM (short username kudu)
I20260808 16:52:57.483295 24687 webserver.cc:460] Webserver started at http://127.8.24.1:41871/ using document root <none> and password file <none>
I20260808 16:52:57.483469 24687 fs_manager.cc:359] Metadata directory not provided
I20260808 16:52:57.483505 24687 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:52:57.483592 24687 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:52:57.484509 24687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786207919384338-8288-0/minicluster-data/ts-0/data/instance:
uuid: "c8616d3510ea40428e98a492974ed0b0"
format_stamp: "Formatted at 2026-08-08 16:52:57 on dist-test-slave-6zbq"
server_key: "b02c8dd61749d97e02ab73abfcbdc10f"
server_key_iv: "d0ba6ae739e3dec6276b9f8139658cc7"
server_key_version: "encryptionkey@0"
I20260808 16:52:57.484799 24687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786207919384338-8288-0/minicluster-data/ts-0/wal/instance:
uuid: "c8616d3510ea40428e98a492974ed0b0"
format_stamp: "Formatted at 2026-08-08 16:52:57 on dist-test-slave-6zbq"
server_key: "b02c8dd61749d97e02ab73abfcbdc10f"
server_key_iv: "d0ba6ae739e3dec6276b9f8139658cc7"
server_key_version: "encryptionkey@0"
I20260808 16:52:57.485934 24687 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 16:52:57.486575 24709 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:57.486722 24687 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:57.486802 24687 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786207919384338-8288-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786207919384338-8288-0/minicluster-data/ts-0/wal
uuid: "c8616d3510ea40428e98a492974ed0b0"
format_stamp: "Formatted at 2026-08-08 16:52:57 on dist-test-slave-6zbq"
server_key: "b02c8dd61749d97e02ab73abfcbdc10f"
server_key_iv: "d0ba6ae739e3dec6276b9f8139658cc7"
server_key_version: "encryptionkey@0"
I20260808 16:52:57.486861 24687 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786207919384338-8288-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786207919384338-8288-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786207919384338-8288-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:52:57.495260 24687 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:52:57.497165 24687 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:52:57.497296 24687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:52:57.497489 24687 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:52:57.497763 24687 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:52:57.497802 24687 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:57.497828 24687 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:52:57.497852 24687 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:57.503144 24687 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.1:35569
I20260808 16:52:57.503198 24876 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.1:35569 every 8 connection(s)
I20260808 16:52:57.503456 24687 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786207919384338-8288-0/minicluster-data/ts-0/data/info.pb
I20260808 16:52:57.507519 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 24687
I20260808 16:52:57.507606 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786207919384338-8288-0/minicluster-data/ts-0/wal/instance
I20260808 16:52:57.507794 8288 external_mini_cluster.cc:1428] Setting key 9a06a7fc3d63f35428815981d697eb25
Aug 08 16:52:57 dist-test-slave-6zbq krb5kdc[24526](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207977, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.1@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260808 16:52:57.512040 24654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:57.504012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:36617 (local address 127.8.24.62:41097)
0808 16:52:57.504054 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:57.504055 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:57.504233 (+ 178us) server_negotiation.cc:407] Connection header received
0808 16:52:57.504717 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:57.504718 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:57.504732 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:57.504783 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:57.505623 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:57.506103 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:57.507086 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:57.507287 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:57.507342 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:57.509473 (+ 2131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:57.509474 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:57.509474 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:57.509481 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:57.510831 (+ 1350us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:57.511214 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:57.511215 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:57.511216 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:57.511275 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:57.511514 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:57.511515 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:57.511515 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:57.511605 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:57.511648 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:57.511918 (+ 270us) server_negotiation.cc:299] Negotiation successful
0808 16:52:57.511980 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 16:52:57.512235 24882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:57.503982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:41097 (local address 127.8.24.1:36617)
0808 16:52:57.504135 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:57.504168 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:57.504616 (+ 448us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:57.504854 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:57.504858 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:57.505368 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:57.505533 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:57.505537 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:57.506187 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:57.506188 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:57.507004 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:57.507007 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:57.507357 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:57.507358 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:57.507491 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:57.508004 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:57.508014 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:57.509393 (+ 1379us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:52:57.510906 (+ 1513us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:57.510913 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:57.510914 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:57.511117 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:57.511336 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:57.511336 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:57.511337 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:57.511458 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:57.511653 (+ 195us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:52:57.511658 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:52:57.511800 (+ 142us) client_negotiation.cc:769] Sending connection context
0808 16:52:57.511896 (+ 96us) client_negotiation.cc:240] Negotiation successful
0808 16:52:57.512039 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":84,"threads_started":1}
I20260808 16:52:57.512718 24877 heartbeater.cc:347] Connected to a master server at 127.8.24.62:41097
I20260808 16:52:57.512812 24877 heartbeater.cc:464] Registering TS with master...
I20260808 16:52:57.512984 24877 heartbeater.cc:511] Master 127.8.24.62:41097 requested a full tablet report, sending...
I20260808 16:52:57.513473 24598 ts_manager.cc:194] Registered new tserver with Master: c8616d3510ea40428e98a492974ed0b0 (127.8.24.1:35569)
I20260808 16:52:57.514039 24598 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.1@KRBTEST.COM'} at 127.8.24.1:36617
WARNING: no policy specified for kudu/127.8.24.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.2.keytab.
Entry for principal kudu/127.8.24.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.2.keytab.
I20260808 16:52:57.530712 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786207919384338-8288-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786207919384338-8288-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786207919384338-8288-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786207919384338-8288-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.2:0
--local_ip_for_outbound_sockets=127.8.24.2
--webserver_interface=127.8.24.2
--webserver_port=0
--tserver_master_addrs=127.8.24.62:41097
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.2.keytab
--principal=kudu/127.8.24.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:33347
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:52:57.617097 24889 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:52:57.617285 24889 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:52:57.617314 24889 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:52:57.618438 24889 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:52:57.618476 24889 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:52:57.618513 24889 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.2
I20260808 16:52:57.619870 24889 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:33347
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786207919384338-8288-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786207919384338-8288-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.2.keytab
--principal=kudu/127.8.24.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786207919384338-8288-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.24.2
--webserver_port=0
--tserver_master_addrs=127.8.24.62:41097
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24889
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.2
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786207919384338-8288-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:52:57.620077 24889 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:52:57.620270 24889 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:52:57.624692 24901 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:57.624692 24899 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:57.624859 24889 server_base.cc:1034] running on GCE node
W20260808 16:52:57.624994 24903 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:57.625188 24889 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:52:57.625461 24889 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:52:57.626621 24889 hybrid_clock.cc:648] HybridClock initialized: now 1786207977626602 us; error 33 us; skew 500 ppm
Aug 08 16:52:57 dist-test-slave-6zbq krb5kdc[24526](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207977, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:52:57.629370 24889 init.cc:377] Logged in from keytab as kudu/127.8.24.2@KRBTEST.COM (short username kudu)
I20260808 16:52:57.629738 24889 webserver.cc:460] Webserver started at http://127.8.24.2:35471/ using document root <none> and password file <none>
I20260808 16:52:57.629916 24889 fs_manager.cc:359] Metadata directory not provided
I20260808 16:52:57.629962 24889 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:52:57.630079 24889 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:52:57.631032 24889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786207919384338-8288-0/minicluster-data/ts-1/data/instance:
uuid: "dec94c487df54d0fa23139ebda3881d4"
format_stamp: "Formatted at 2026-08-08 16:52:57 on dist-test-slave-6zbq"
server_key: "f56de356707768f0945b0b3fc76c01b4"
server_key_iv: "90ea0e15ed2ee297b7aaefe733d2d079"
server_key_version: "encryptionkey@0"
I20260808 16:52:57.631346 24889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786207919384338-8288-0/minicluster-data/ts-1/wal/instance:
uuid: "dec94c487df54d0fa23139ebda3881d4"
format_stamp: "Formatted at 2026-08-08 16:52:57 on dist-test-slave-6zbq"
server_key: "f56de356707768f0945b0b3fc76c01b4"
server_key_iv: "90ea0e15ed2ee297b7aaefe733d2d079"
server_key_version: "encryptionkey@0"
I20260808 16:52:57.632478 24889 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 16:52:57.633068 24917 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:57.633234 24889 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 16:52:57.633314 24889 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786207919384338-8288-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786207919384338-8288-0/minicluster-data/ts-1/wal
uuid: "dec94c487df54d0fa23139ebda3881d4"
format_stamp: "Formatted at 2026-08-08 16:52:57 on dist-test-slave-6zbq"
server_key: "f56de356707768f0945b0b3fc76c01b4"
server_key_iv: "90ea0e15ed2ee297b7aaefe733d2d079"
server_key_version: "encryptionkey@0"
I20260808 16:52:57.633370 24889 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786207919384338-8288-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786207919384338-8288-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786207919384338-8288-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:52:57.644428 24889 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:52:57.645951 24889 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:52:57.646073 24889 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:52:57.646256 24889 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:52:57.646584 24889 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:52:57.646618 24889 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:57.646643 24889 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:52:57.646664 24889 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:57.651839 24889 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.2:46797
I20260808 16:52:57.651875 25103 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.2:46797 every 8 connection(s)
I20260808 16:52:57.652125 24889 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786207919384338-8288-0/minicluster-data/ts-1/data/info.pb
I20260808 16:52:57.655329 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 24889
I20260808 16:52:57.655428 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786207919384338-8288-0/minicluster-data/ts-1/wal/instance
I20260808 16:52:57.655613 8288 external_mini_cluster.cc:1428] Setting key df47c97c5a5d42dabe712115ed462b9e
Aug 08 16:52:57 dist-test-slave-6zbq krb5kdc[24526](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207977, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.2@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260808 16:52:57.660817 24654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:57.652722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:57217 (local address 127.8.24.62:41097)
0808 16:52:57.652767 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:57.652770 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:57.652944 (+ 174us) server_negotiation.cc:407] Connection header received
0808 16:52:57.653344 (+ 400us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:57.653345 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:57.653355 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:57.653402 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:57.654205 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:57.654697 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:57.655876 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:57.656126 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:57.656197 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:57.658281 (+ 2084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:57.658282 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:57.658282 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:57.658289 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:57.659644 (+ 1355us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:57.659989 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:57.659991 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:57.659991 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:57.660052 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:57.660253 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:57.660254 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:57.660255 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:57.660391 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:57.660461 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:57.660643 (+ 182us) server_negotiation.cc:299] Negotiation successful
0808 16:52:57.660729 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 16:52:57.660933 25109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:57.652700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:41097 (local address 127.8.24.2:57217)
0808 16:52:57.652843 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:57.652874 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:57.653273 (+ 399us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:57.653457 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:57.653460 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:57.653973 (+ 513us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:57.654139 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:57.654143 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:57.654764 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:57.654765 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:57.655793 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:57.655797 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:57.656205 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:57.656206 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:57.656338 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:57.656751 (+ 413us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:57.656760 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:57.658197 (+ 1437us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:52:57.659717 (+ 1520us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:57.659722 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:57.659724 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:57.659925 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:57.660111 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:57.660111 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:57.660112 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:57.660194 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:57.660449 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:52:57.660452 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:52:57.660545 (+ 93us) client_negotiation.cc:769] Sending connection context
0808 16:52:57.660613 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 16:52:57.660715 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":75,"threads_started":1}
I20260808 16:52:57.661348 25104 heartbeater.cc:347] Connected to a master server at 127.8.24.62:41097
I20260808 16:52:57.661427 25104 heartbeater.cc:464] Registering TS with master...
I20260808 16:52:57.661572 25104 heartbeater.cc:511] Master 127.8.24.62:41097 requested a full tablet report, sending...
I20260808 16:52:57.661917 24598 ts_manager.cc:194] Registered new tserver with Master: dec94c487df54d0fa23139ebda3881d4 (127.8.24.2:46797)
I20260808 16:52:57.662422 24598 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.2@KRBTEST.COM'} at 127.8.24.2:57217
WARNING: no policy specified for kudu/127.8.24.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.3.keytab.
Entry for principal kudu/127.8.24.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.3.keytab.
I20260808 16:52:57.679143 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786207919384338-8288-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786207919384338-8288-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786207919384338-8288-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786207919384338-8288-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.3:0
--local_ip_for_outbound_sockets=127.8.24.3
--webserver_interface=127.8.24.3
--webserver_port=0
--tserver_master_addrs=127.8.24.62:41097
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.3.keytab
--principal=kudu/127.8.24.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:33347
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:52:57.756150 25112 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:52:57.756331 25112 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:52:57.756352 25112 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:52:57.757400 25112 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:52:57.757431 25112 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:52:57.757454 25112 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.3
I20260808 16:52:57.758777 25112 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:33347
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786207919384338-8288-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786207919384338-8288-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.3.keytab
--principal=kudu/127.8.24.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786207919384338-8288-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.24.3
--webserver_port=0
--tserver_master_addrs=127.8.24.62:41097
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25112
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.3
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786207919384338-8288-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:52:57.758934 25112 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:52:57.759094 25112 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:52:57.763507 25118 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:52:57.763588 25117 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:57.763564 25112 server_base.cc:1034] running on GCE node
W20260808 16:52:57.763679 25122 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:52:57.763862 25112 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:52:57.764078 25112 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:52:57.765208 25112 hybrid_clock.cc:648] HybridClock initialized: now 1786207977765193 us; error 27 us; skew 500 ppm
Aug 08 16:52:57 dist-test-slave-6zbq krb5kdc[24526](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207977, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:52:57.767431 25112 init.cc:377] Logged in from keytab as kudu/127.8.24.3@KRBTEST.COM (short username kudu)
I20260808 16:52:57.767805 25112 webserver.cc:460] Webserver started at http://127.8.24.3:33385/ using document root <none> and password file <none>
I20260808 16:52:57.767985 25112 fs_manager.cc:359] Metadata directory not provided
I20260808 16:52:57.768029 25112 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:52:57.768147 25112 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:52:57.769060 25112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786207919384338-8288-0/minicluster-data/ts-2/data/instance:
uuid: "bf3fc5736a524c77b298cb55de1d4d96"
format_stamp: "Formatted at 2026-08-08 16:52:57 on dist-test-slave-6zbq"
server_key: "e2fb07805d044fbcf4993395eced3d86"
server_key_iv: "930e15e486bd0494b3c3917519060377"
server_key_version: "encryptionkey@0"
I20260808 16:52:57.769373 25112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786207919384338-8288-0/minicluster-data/ts-2/wal/instance:
uuid: "bf3fc5736a524c77b298cb55de1d4d96"
format_stamp: "Formatted at 2026-08-08 16:52:57 on dist-test-slave-6zbq"
server_key: "e2fb07805d044fbcf4993395eced3d86"
server_key_iv: "930e15e486bd0494b3c3917519060377"
server_key_version: "encryptionkey@0"
I20260808 16:52:57.770570 25112 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 16:52:57.771186 25130 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:57.771334 25112 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:57.771411 25112 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786207919384338-8288-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786207919384338-8288-0/minicluster-data/ts-2/wal
uuid: "bf3fc5736a524c77b298cb55de1d4d96"
format_stamp: "Formatted at 2026-08-08 16:52:57 on dist-test-slave-6zbq"
server_key: "e2fb07805d044fbcf4993395eced3d86"
server_key_iv: "930e15e486bd0494b3c3917519060377"
server_key_version: "encryptionkey@0"
I20260808 16:52:57.771468 25112 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786207919384338-8288-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786207919384338-8288-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786207919384338-8288-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:52:57.778813 25112 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:52:57.780303 25112 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:52:57.780422 25112 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:52:57.780594 25112 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:52:57.780853 25112 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:52:57.780886 25112 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:57.780911 25112 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:52:57.780936 25112 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:52:57.786090 25112 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.3:37939
I20260808 16:52:57.786142 25304 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.3:37939 every 8 connection(s)
I20260808 16:52:57.786437 25112 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786207919384338-8288-0/minicluster-data/ts-2/data/info.pb
Aug 08 16:52:57 dist-test-slave-6zbq krb5kdc[24526](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207977, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.3@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260808 16:52:57.793115 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 25112
I20260808 16:52:57.793205 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786207919384338-8288-0/minicluster-data/ts-2/wal/instance
I20260808 16:52:57.793411 8288 external_mini_cluster.cc:1428] Setting key c8d12daa772e6596deb319bfc6c717ac
I20260808 16:52:57.794605 24654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:57.787050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:35701 (local address 127.8.24.62:41097)
0808 16:52:57.787098 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:57.787100 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:57.787269 (+ 169us) server_negotiation.cc:407] Connection header received
0808 16:52:57.787699 (+ 430us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:57.787701 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:57.787713 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:57.787762 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:57.788583 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:57.789124 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:57.790092 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:57.790290 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:57.790365 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:57.792146 (+ 1781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:57.792147 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:57.792148 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:57.792155 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:57.793220 (+ 1065us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:57.793686 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:57.793688 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:57.793688 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:57.793742 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:57.793984 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:57.793985 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:57.793985 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:57.794091 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:57.794147 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:57.794441 (+ 294us) server_negotiation.cc:299] Negotiation successful
0808 16:52:57.794519 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 16:52:57.794737 25308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:57.787008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:41097 (local address 127.8.24.3:35701)
0808 16:52:57.787169 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:57.787201 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:57.787613 (+ 412us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:57.787822 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:57.787826 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:57.788327 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:52:57.788495 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:57.788500 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:57.789203 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:57.789204 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:57.790023 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:57.790026 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:57.790377 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:57.790378 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:57.790489 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:57.790881 (+ 392us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:52:57.790889 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:52:57.792064 (+ 1175us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:52:57.793299 (+ 1235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:57.793307 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:57.793308 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:57.793590 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:57.793794 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:52:57.793794 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:52:57.793795 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:52:57.793921 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:52:57.794165 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:52:57.794172 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:52:57.794299 (+ 127us) client_negotiation.cc:769] Sending connection context
0808 16:52:57.794405 (+ 106us) client_negotiation.cc:240] Negotiation successful
0808 16:52:57.794545 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":87,"threads_started":1}
I20260808 16:52:57.795113 25306 heartbeater.cc:347] Connected to a master server at 127.8.24.62:41097
I20260808 16:52:57.795193 25306 heartbeater.cc:464] Registering TS with master...
I20260808 16:52:57.795348 25306 heartbeater.cc:511] Master 127.8.24.62:41097 requested a full tablet report, sending...
I20260808 16:52:57.795689 24598 ts_manager.cc:194] Registered new tserver with Master: bf3fc5736a524c77b298cb55de1d4d96 (127.8.24.3:37939)
I20260808 16:52:57.796037 24598 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.3@KRBTEST.COM'} at 127.8.24.3:35701
I20260808 16:52:57.804807 8288 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 16:52:57.810976 24654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:57.805989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42826 (local address 127.8.24.62:41097)
0808 16:52:57.806026 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:57.806028 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:57.806132 (+ 104us) server_negotiation.cc:407] Connection header received
0808 16:52:57.806185 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:57.806186 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:57.806195 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:57.806237 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:52:57.806789 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:57.807223 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:57.807897 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:57.808064 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:57.808114 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:57.809066 (+ 952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:52:57.809067 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:52:57.809067 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:52:57.809073 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:52:57.809989 (+ 916us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:57.810274 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:57.810275 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:57.810275 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:57.810322 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:52:57.810503 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:52:57.810504 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:52:57.810504 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:52:57.810631 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:52:57.810711 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:57.810830 (+ 119us) server_negotiation.cc:299] Negotiation successful
0808 16:52:57.810895 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260808 16:52:57.812211 24598 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42826:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 16:52:57.813637 24598 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 16:52:57.819471 25324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:57.815710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50814 (local address 127.8.24.3:37939)
0808 16:52:57.815835 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:57.815839 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:57.816113 (+ 274us) server_negotiation.cc:407] Connection header received
0808 16:52:57.816411 (+ 298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:57.816416 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:57.816433 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:57.816505 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:52:57.816675 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:57.817645 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:57.818677 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:57.819120 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:57.819189 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:57.819243 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:57.819367 (+ 124us) server_negotiation.cc:299] Negotiation successful
0808 16:52:57.819403 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":57,"threads_started":1}
I20260808 16:52:57.819512 25322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:57.815450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:35569 (local address 127.0.0.1:37294)
0808 16:52:57.815846 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:57.815883 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:57.815946 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:57.816122 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:57.816140 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:57.816142 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:52:57.816198 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:57.816199 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:57.817188 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:57.817188 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:57.818265 (+ 1077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:57.818267 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:57.819238 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:57.819239 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:57.819301 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:57.819305 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 16:52:57.819351 (+ 46us) client_negotiation.cc:240] Negotiation successful
0808 16:52:57.819427 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":49,"threads_started":1}
I20260808 16:52:57.819519 25321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:57.815617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.3:37939 (local address 127.0.0.1:50814)
0808 16:52:57.816058 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:57.816066 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:57.816137 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:57.816557 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:57.816558 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:57.816560 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:52:57.816600 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:57.816602 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:57.817730 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:57.817732 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:57.818606 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:57.818609 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:57.819194 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:57.819195 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:57.819272 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:57.819285 (+ 13us) client_negotiation.cc:769] Sending connection context
0808 16:52:57.819348 (+ 63us) client_negotiation.cc:240] Negotiation successful
0808 16:52:57.819432 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":434,"mutex_wait_us":38,"thread_start_us":52,"threads_started":1}
I20260808 16:52:57.819818 25326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:57.815782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37294 (local address 127.8.24.1:35569)
0808 16:52:57.815937 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:57.815940 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:57.815951 (+ 11us) server_negotiation.cc:407] Connection header received
0808 16:52:57.816031 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:57.816033 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:57.816048 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:57.816115 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:52:57.816281 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:57.817098 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:57.818381 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:57.819078 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:57.819139 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:57.819202 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:57.819693 (+ 491us) server_negotiation.cc:299] Negotiation successful
0808 16:52:57.819738 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":77,"threads_started":1}
I20260808 16:52:57.820022 25323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:57.815572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46884 (local address 127.8.24.2:46797)
0808 16:52:57.815744 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:57.815748 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:57.816000 (+ 252us) server_negotiation.cc:407] Connection header received
0808 16:52:57.816050 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:57.816053 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:57.816069 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:57.816142 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:52:57.816371 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:57.817446 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:57.818718 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:57.819388 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:57.819443 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:57.819493 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:57.819929 (+ 436us) server_negotiation.cc:299] Negotiation successful
0808 16:52:57.819966 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":107,"threads_started":1}
I20260808 16:52:57.820201 25317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:57.815470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.2:46797 (local address 127.0.0.1:46884)
0808 16:52:57.815923 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:57.815931 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:57.815988 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:57.816254 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:57.816255 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:57.816257 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:52:57.816302 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:57.816303 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:57.817551 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:57.817552 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:57.818621 (+ 1069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:57.818624 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:57.819652 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:57.819653 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:57.819723 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:57.819726 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 16:52:57.820088 (+ 362us) client_negotiation.cc:240] Negotiation successful
0808 16:52:57.820136 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":441,"mutex_wait_us":1,"thread_start_us":51,"threads_started":1}
I20260808 16:52:57.820372 25205 tablet_service.cc:1467] Processing CreateTablet for tablet 81699f3c6f0241479f1d266a19359742 (DEFAULT_TABLE table=test-table [id=3488af28dc1243559feec270182e03d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:52:57.820478 24782 tablet_service.cc:1467] Processing CreateTablet for tablet 81699f3c6f0241479f1d266a19359742 (DEFAULT_TABLE table=test-table [id=3488af28dc1243559feec270182e03d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:52:57.820732 25205 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81699f3c6f0241479f1d266a19359742. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:52:57.820758 24782 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81699f3c6f0241479f1d266a19359742. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:52:57.822826 24990 tablet_service.cc:1467] Processing CreateTablet for tablet 81699f3c6f0241479f1d266a19359742 (DEFAULT_TABLE table=test-table [id=3488af28dc1243559feec270182e03d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:52:57.823104 24990 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81699f3c6f0241479f1d266a19359742. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:52:57.823894 25330 tablet_bootstrap.cc:492] T 81699f3c6f0241479f1d266a19359742 P bf3fc5736a524c77b298cb55de1d4d96: Bootstrap starting.
I20260808 16:52:57.824095 25332 tablet_bootstrap.cc:492] T 81699f3c6f0241479f1d266a19359742 P c8616d3510ea40428e98a492974ed0b0: Bootstrap starting.
I20260808 16:52:57.824754 25332 tablet_bootstrap.cc:654] T 81699f3c6f0241479f1d266a19359742 P c8616d3510ea40428e98a492974ed0b0: Neither blocks nor log segments found. Creating new log.
I20260808 16:52:57.825299 25332 log.cc:824] T 81699f3c6f0241479f1d266a19359742 P c8616d3510ea40428e98a492974ed0b0: Log is configured to *not* fsync() on all Append() calls
I20260808 16:52:57.825964 25332 tablet_bootstrap.cc:492] T 81699f3c6f0241479f1d266a19359742 P c8616d3510ea40428e98a492974ed0b0: No bootstrap required, opened a new log
I20260808 16:52:57.826027 25332 ts_tablet_manager.cc:1397] T 81699f3c6f0241479f1d266a19359742 P c8616d3510ea40428e98a492974ed0b0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 16:52:57.826105 25330 tablet_bootstrap.cc:654] T 81699f3c6f0241479f1d266a19359742 P bf3fc5736a524c77b298cb55de1d4d96: Neither blocks nor log segments found. Creating new log.
I20260808 16:52:57.826225 25335 tablet_bootstrap.cc:492] T 81699f3c6f0241479f1d266a19359742 P dec94c487df54d0fa23139ebda3881d4: Bootstrap starting.
I20260808 16:52:57.826884 25335 tablet_bootstrap.cc:654] T 81699f3c6f0241479f1d266a19359742 P dec94c487df54d0fa23139ebda3881d4: Neither blocks nor log segments found. Creating new log.
I20260808 16:52:57.826905 25330 log.cc:824] T 81699f3c6f0241479f1d266a19359742 P bf3fc5736a524c77b298cb55de1d4d96: Log is configured to *not* fsync() on all Append() calls
I20260808 16:52:57.827394 25335 log.cc:824] T 81699f3c6f0241479f1d266a19359742 P dec94c487df54d0fa23139ebda3881d4: Log is configured to *not* fsync() on all Append() calls
I20260808 16:52:57.827510 25330 tablet_bootstrap.cc:492] T 81699f3c6f0241479f1d266a19359742 P bf3fc5736a524c77b298cb55de1d4d96: No bootstrap required, opened a new log
I20260808 16:52:57.827576 25330 ts_tablet_manager.cc:1397] T 81699f3c6f0241479f1d266a19359742 P bf3fc5736a524c77b298cb55de1d4d96: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20260808 16:52:57.828022 25332 raft_consensus.cc:348] T 81699f3c6f0241479f1d266a19359742 P c8616d3510ea40428e98a492974ed0b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8616d3510ea40428e98a492974ed0b0" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 35569 } } peers { permanent_uuid: "bf3fc5736a524c77b298cb55de1d4d96" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 37939 } } peers { permanent_uuid: "dec94c487df54d0fa23139ebda3881d4" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 46797 } }
I20260808 16:52:57.828137 25335 tablet_bootstrap.cc:492] T 81699f3c6f0241479f1d266a19359742 P dec94c487df54d0fa23139ebda3881d4: No bootstrap required, opened a new log
I20260808 16:52:57.828154 25332 raft_consensus.cc:374] T 81699f3c6f0241479f1d266a19359742 P c8616d3510ea40428e98a492974ed0b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:52:57.828192 25335 ts_tablet_manager.cc:1397] T 81699f3c6f0241479f1d266a19359742 P dec94c487df54d0fa23139ebda3881d4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 16:52:57.828190 25332 raft_consensus.cc:729] T 81699f3c6f0241479f1d266a19359742 P c8616d3510ea40428e98a492974ed0b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8616d3510ea40428e98a492974ed0b0, State: Initialized, Role: FOLLOWER
I20260808 16:52:57.828323 25332 consensus_queue.cc:260] T 81699f3c6f0241479f1d266a19359742 P c8616d3510ea40428e98a492974ed0b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8616d3510ea40428e98a492974ed0b0" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 35569 } } peers { permanent_uuid: "bf3fc5736a524c77b298cb55de1d4d96" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 37939 } } peers { permanent_uuid: "dec94c487df54d0fa23139ebda3881d4" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 46797 } }
I20260808 16:52:57.828588 24877 heartbeater.cc:503] Master 127.8.24.62:41097 was elected leader, sending a full tablet report...
I20260808 16:52:57.828585 25332 ts_tablet_manager.cc:1428] T 81699f3c6f0241479f1d266a19359742 P c8616d3510ea40428e98a492974ed0b0: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 16:52:57.829311 25330 raft_consensus.cc:348] T 81699f3c6f0241479f1d266a19359742 P bf3fc5736a524c77b298cb55de1d4d96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8616d3510ea40428e98a492974ed0b0" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 35569 } } peers { permanent_uuid: "bf3fc5736a524c77b298cb55de1d4d96" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 37939 } } peers { permanent_uuid: "dec94c487df54d0fa23139ebda3881d4" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 46797 } }
I20260808 16:52:57.829460 25330 raft_consensus.cc:374] T 81699f3c6f0241479f1d266a19359742 P bf3fc5736a524c77b298cb55de1d4d96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:52:57.829495 25330 raft_consensus.cc:729] T 81699f3c6f0241479f1d266a19359742 P bf3fc5736a524c77b298cb55de1d4d96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf3fc5736a524c77b298cb55de1d4d96, State: Initialized, Role: FOLLOWER
I20260808 16:52:57.829608 25330 consensus_queue.cc:260] T 81699f3c6f0241479f1d266a19359742 P bf3fc5736a524c77b298cb55de1d4d96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8616d3510ea40428e98a492974ed0b0" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 35569 } } peers { permanent_uuid: "bf3fc5736a524c77b298cb55de1d4d96" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 37939 } } peers { permanent_uuid: "dec94c487df54d0fa23139ebda3881d4" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 46797 } }
I20260808 16:52:57.829878 25330 ts_tablet_manager.cc:1428] T 81699f3c6f0241479f1d266a19359742 P bf3fc5736a524c77b298cb55de1d4d96: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260808 16:52:57.829895 25306 heartbeater.cc:503] Master 127.8.24.62:41097 was elected leader, sending a full tablet report...
I20260808 16:52:57.830010 25335 raft_consensus.cc:348] T 81699f3c6f0241479f1d266a19359742 P dec94c487df54d0fa23139ebda3881d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8616d3510ea40428e98a492974ed0b0" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 35569 } } peers { permanent_uuid: "bf3fc5736a524c77b298cb55de1d4d96" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 37939 } } peers { permanent_uuid: "dec94c487df54d0fa23139ebda3881d4" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 46797 } }
I20260808 16:52:57.830127 25335 raft_consensus.cc:374] T 81699f3c6f0241479f1d266a19359742 P dec94c487df54d0fa23139ebda3881d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:52:57.830156 25335 raft_consensus.cc:729] T 81699f3c6f0241479f1d266a19359742 P dec94c487df54d0fa23139ebda3881d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dec94c487df54d0fa23139ebda3881d4, State: Initialized, Role: FOLLOWER
I20260808 16:52:57.830248 25335 consensus_queue.cc:260] T 81699f3c6f0241479f1d266a19359742 P dec94c487df54d0fa23139ebda3881d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8616d3510ea40428e98a492974ed0b0" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 35569 } } peers { permanent_uuid: "bf3fc5736a524c77b298cb55de1d4d96" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 37939 } } peers { permanent_uuid: "dec94c487df54d0fa23139ebda3881d4" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 46797 } }
I20260808 16:52:57.830524 25335 ts_tablet_manager.cc:1428] T 81699f3c6f0241479f1d266a19359742 P dec94c487df54d0fa23139ebda3881d4: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 16:52:57.830633 25104 heartbeater.cc:503] Master 127.8.24.62:41097 was elected leader, sending a full tablet report...
W20260808 16:52:57.902572 25105 tablet.cc:2369] T 81699f3c6f0241479f1d266a19359742 P dec94c487df54d0fa23139ebda3881d4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 16:52:58.004114 24880 tablet.cc:2369] T 81699f3c6f0241479f1d266a19359742 P c8616d3510ea40428e98a492974ed0b0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 16:52:58.036911 25307 tablet.cc:2369] T 81699f3c6f0241479f1d266a19359742 P bf3fc5736a524c77b298cb55de1d4d96: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 16:52:58.049118 25341 raft_consensus.cc:482] T 81699f3c6f0241479f1d266a19359742 P dec94c487df54d0fa23139ebda3881d4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 16:52:58.049216 25341 raft_consensus.cc:504] T 81699f3c6f0241479f1d266a19359742 P dec94c487df54d0fa23139ebda3881d4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8616d3510ea40428e98a492974ed0b0" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 35569 } } peers { permanent_uuid: "bf3fc5736a524c77b298cb55de1d4d96" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 37939 } } peers { permanent_uuid: "dec94c487df54d0fa23139ebda3881d4" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 46797 } }
I20260808 16:52:58.049566 25341 leader_election.cc:288] T 81699f3c6f0241479f1d266a19359742 P dec94c487df54d0fa23139ebda3881d4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8616d3510ea40428e98a492974ed0b0 (127.8.24.1:35569), bf3fc5736a524c77b298cb55de1d4d96 (127.8.24.3:37939)
I20260808 16:52:58.052132 25109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:58.049713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:35569 (local address 127.8.24.2:46929)
0808 16:52:58.049769 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:58.049780 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:58.049822 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:58.049964 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:58.049966 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:58.049968 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:52:58.050037 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:58.050039 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:58.050672 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:58.050674 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:58.051397 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:58.051399 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:58.051892 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:58.051893 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:58.051967 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:58.051970 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 16:52:58.052015 (+ 45us) client_negotiation.cc:240] Negotiation successful
0808 16:52:58.052052 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260808 16:52:58.052124 25326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:58.049769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:46929 (local address 127.8.24.1:35569)
0808 16:52:58.049821 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:58.049824 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:58.049834 (+ 10us) server_negotiation.cc:407] Connection header received
0808 16:52:58.049883 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:58.049884 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:58.049896 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:58.049954 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:52:58.050103 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:58.050594 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:58.051476 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:58.051821 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:58.051880 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:58.051891 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:58.052026 (+ 135us) server_negotiation.cc:299] Negotiation successful
0808 16:52:58.052060 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 16:52:58.052280 25324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:58.049901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:44441 (local address 127.8.24.3:37939)
0808 16:52:58.049963 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:58.049965 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:58.050050 (+ 85us) server_negotiation.cc:407] Connection header received
0808 16:52:58.050095 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:58.050096 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:58.050108 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:58.050160 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:52:58.050294 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:58.050763 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:58.051626 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:58.051980 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:58.052042 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:58.052051 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:58.052189 (+ 138us) server_negotiation.cc:299] Negotiation successful
0808 16:52:58.052225 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 16:52:58.052284 25342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:58.049827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.3:37939 (local address 127.8.24.2:44441)
0808 16:52:58.049991 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:58.050000 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:58.050048 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:58.050160 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:52:58.050160 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:52:58.050162 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:52:58.050238 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:58.050240 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:58.050829 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:58.050830 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:58.051544 (+ 714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:52:58.051546 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:52:58.052051 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:52:58.052053 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:52:58.052120 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:58.052123 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 16:52:58.052167 (+ 44us) client_negotiation.cc:240] Negotiation successful
0808 16:52:58.052204 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":104,"threads_started":1}
I20260808 16:52:58.052454 24813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81699f3c6f0241479f1d266a19359742" candidate_uuid: "dec94c487df54d0fa23139ebda3881d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8616d3510ea40428e98a492974ed0b0" is_pre_election: true
I20260808 16:52:58.052598 24813 raft_consensus.cc:2454] T 81699f3c6f0241479f1d266a19359742 P c8616d3510ea40428e98a492974ed0b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dec94c487df54d0fa23139ebda3881d4 in term 0.
I20260808 16:52:58.052811 24921 leader_election.cc:302] T 81699f3c6f0241479f1d266a19359742 P dec94c487df54d0fa23139ebda3881d4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c8616d3510ea40428e98a492974ed0b0, dec94c487df54d0fa23139ebda3881d4; no voters:
I20260808 16:52:58.052943 25341 raft_consensus.cc:2781] T 81699f3c6f0241479f1d266a19359742 P dec94c487df54d0fa23139ebda3881d4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 16:52:58.052984 25341 raft_consensus.cc:482] T 81699f3c6f0241479f1d266a19359742 P dec94c487df54d0fa23139ebda3881d4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 16:52:58.052930 25237 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81699f3c6f0241479f1d266a19359742" candidate_uuid: "dec94c487df54d0fa23139ebda3881d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf3fc5736a524c77b298cb55de1d4d96" is_pre_election: true
I20260808 16:52:58.053006 25341 raft_consensus.cc:3037] T 81699f3c6f0241479f1d266a19359742 P dec94c487df54d0fa23139ebda3881d4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:52:58.053062 25237 raft_consensus.cc:2454] T 81699f3c6f0241479f1d266a19359742 P bf3fc5736a524c77b298cb55de1d4d96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dec94c487df54d0fa23139ebda3881d4 in term 0.
I20260808 16:52:58.053745 25341 raft_consensus.cc:504] T 81699f3c6f0241479f1d266a19359742 P dec94c487df54d0fa23139ebda3881d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8616d3510ea40428e98a492974ed0b0" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 35569 } } peers { permanent_uuid: "bf3fc5736a524c77b298cb55de1d4d96" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 37939 } } peers { permanent_uuid: "dec94c487df54d0fa23139ebda3881d4" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 46797 } }
I20260808 16:52:58.053854 25341 leader_election.cc:288] T 81699f3c6f0241479f1d266a19359742 P dec94c487df54d0fa23139ebda3881d4 [CANDIDATE]: Term 1 election: Requested vote from peers c8616d3510ea40428e98a492974ed0b0 (127.8.24.1:35569), bf3fc5736a524c77b298cb55de1d4d96 (127.8.24.3:37939)
I20260808 16:52:58.054031 24813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81699f3c6f0241479f1d266a19359742" candidate_uuid: "dec94c487df54d0fa23139ebda3881d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8616d3510ea40428e98a492974ed0b0"
I20260808 16:52:58.054037 25237 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81699f3c6f0241479f1d266a19359742" candidate_uuid: "dec94c487df54d0fa23139ebda3881d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf3fc5736a524c77b298cb55de1d4d96"
I20260808 16:52:58.054100 25237 raft_consensus.cc:3037] T 81699f3c6f0241479f1d266a19359742 P bf3fc5736a524c77b298cb55de1d4d96 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:52:58.054102 24813 raft_consensus.cc:3037] T 81699f3c6f0241479f1d266a19359742 P c8616d3510ea40428e98a492974ed0b0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:52:58.054801 24813 raft_consensus.cc:2454] T 81699f3c6f0241479f1d266a19359742 P c8616d3510ea40428e98a492974ed0b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dec94c487df54d0fa23139ebda3881d4 in term 1.
I20260808 16:52:58.054844 25237 raft_consensus.cc:2454] T 81699f3c6f0241479f1d266a19359742 P bf3fc5736a524c77b298cb55de1d4d96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dec94c487df54d0fa23139ebda3881d4 in term 1.
I20260808 16:52:58.054979 24921 leader_election.cc:302] T 81699f3c6f0241479f1d266a19359742 P dec94c487df54d0fa23139ebda3881d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bf3fc5736a524c77b298cb55de1d4d96, dec94c487df54d0fa23139ebda3881d4; no voters:
I20260808 16:52:58.055080 25341 raft_consensus.cc:2781] T 81699f3c6f0241479f1d266a19359742 P dec94c487df54d0fa23139ebda3881d4 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:52:58.055233 25341 raft_consensus.cc:686] T 81699f3c6f0241479f1d266a19359742 P dec94c487df54d0fa23139ebda3881d4 [term 1 LEADER]: Becoming Leader. State: Replica: dec94c487df54d0fa23139ebda3881d4, State: Running, Role: LEADER
I20260808 16:52:58.055328 25341 consensus_queue.cc:237] T 81699f3c6f0241479f1d266a19359742 P dec94c487df54d0fa23139ebda3881d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8616d3510ea40428e98a492974ed0b0" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 35569 } } peers { permanent_uuid: "bf3fc5736a524c77b298cb55de1d4d96" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 37939 } } peers { permanent_uuid: "dec94c487df54d0fa23139ebda3881d4" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 46797 } }
I20260808 16:52:58.056056 24596 catalog_manager.cc:5495] T 81699f3c6f0241479f1d266a19359742 P dec94c487df54d0fa23139ebda3881d4 reported cstate change: term changed from 0 to 1, leader changed from <none> to dec94c487df54d0fa23139ebda3881d4 (127.8.24.2). New cstate: current_term: 1 leader_uuid: "dec94c487df54d0fa23139ebda3881d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8616d3510ea40428e98a492974ed0b0" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 35569 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf3fc5736a524c77b298cb55de1d4d96" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 37939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dec94c487df54d0fa23139ebda3881d4" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 46797 } health_report { overall_health: HEALTHY } } }
I20260808 16:52:58.097630 25323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:52:58.095656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46894 (local address 127.8.24.2:46797)
0808 16:52:58.095704 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 16:52:58.095705 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 16:52:58.095714 (+ 9us) server_negotiation.cc:407] Connection header received
0808 16:52:58.095744 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:52:58.095745 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:52:58.095760 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:52:58.095818 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 16:52:58.095922 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:58.096389 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:58.097068 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:52:58.097243 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:52:58.097281 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:52:58.097396 (+ 115us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 16:52:58.097480 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 16:52:58.097533 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 16:52:58.097551 (+ 18us) server_negotiation.cc:299] Negotiation successful
0808 16:52:58.097566 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 16:52:58.099193 25237 raft_consensus.cc:1261] T 81699f3c6f0241479f1d266a19359742 P bf3fc5736a524c77b298cb55de1d4d96 [term 1 FOLLOWER]: Refusing update from remote peer dec94c487df54d0fa23139ebda3881d4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:52:58.099203 24813 raft_consensus.cc:1261] T 81699f3c6f0241479f1d266a19359742 P c8616d3510ea40428e98a492974ed0b0 [term 1 FOLLOWER]: Refusing update from remote peer dec94c487df54d0fa23139ebda3881d4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:52:58.099486 25341 consensus_queue.cc:1035] T 81699f3c6f0241479f1d266a19359742 P dec94c487df54d0fa23139ebda3881d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8616d3510ea40428e98a492974ed0b0" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 35569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:52:58.099623 25343 consensus_queue.cc:1035] T 81699f3c6f0241479f1d266a19359742 P dec94c487df54d0fa23139ebda3881d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf3fc5736a524c77b298cb55de1d4d96" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 37939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:52:58.102188 25347 mvcc.cc:205] Tried to move back new op lower bound from 7316307878291939328 to 7316307878115004416. Current Snapshot: MvccSnapshot[applied={T|T < 7316307878291939328}]
I20260808 16:52:58.102293 25350 mvcc.cc:205] Tried to move back new op lower bound from 7316307878291939328 to 7316307878115004416. Current Snapshot: MvccSnapshot[applied={T|T < 7316307878291939328}]
I20260808 16:52:58.103927 24596 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42826:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 16:52:58.104027 24596 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42826:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 16:52:58.104698 24596 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cade7cc8ece44438802332a59875f2b5: Sending DeleteTablet for 3 replicas of tablet 81699f3c6f0241479f1d266a19359742
I20260808 16:52:58.104998 24782 tablet_service.cc:1514] Processing DeleteTablet for tablet 81699f3c6f0241479f1d266a19359742 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:52:58 UTC) from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:37294
I20260808 16:52:58.105011 24990 tablet_service.cc:1514] Processing DeleteTablet for tablet 81699f3c6f0241479f1d266a19359742 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:52:58 UTC) from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:46884
I20260808 16:52:58.105041 25205 tablet_service.cc:1514] Processing DeleteTablet for tablet 81699f3c6f0241479f1d266a19359742 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:52:58 UTC) from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:50814
I20260808 16:52:58.105242 25362 tablet_replica.cc:330] T 81699f3c6f0241479f1d266a19359742 P dec94c487df54d0fa23139ebda3881d4: stopping tablet replica
I20260808 16:52:58.105316 25361 tablet_replica.cc:330] T 81699f3c6f0241479f1d266a19359742 P c8616d3510ea40428e98a492974ed0b0: stopping tablet replica
I20260808 16:52:58.105372 25362 raft_consensus.cc:2229] T 81699f3c6f0241479f1d266a19359742 P dec94c487df54d0fa23139ebda3881d4 [term 1 LEADER]: Raft consensus shutting down.
I20260808 16:52:58.105403 25361 raft_consensus.cc:2229] T 81699f3c6f0241479f1d266a19359742 P c8616d3510ea40428e98a492974ed0b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 16:52:58.105479 25362 raft_consensus.cc:2258] T 81699f3c6f0241479f1d266a19359742 P dec94c487df54d0fa23139ebda3881d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:52:58.105506 25361 raft_consensus.cc:2258] T 81699f3c6f0241479f1d266a19359742 P c8616d3510ea40428e98a492974ed0b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:52:58.105680 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 24687
I20260808 16:52:58.105711 25366 tablet_replica.cc:330] T 81699f3c6f0241479f1d266a19359742 P bf3fc5736a524c77b298cb55de1d4d96: stopping tablet replica
I20260808 16:52:58.105791 25366 raft_consensus.cc:2229] T 81699f3c6f0241479f1d266a19359742 P bf3fc5736a524c77b298cb55de1d4d96 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 16:52:58.105873 25366 raft_consensus.cc:2258] T 81699f3c6f0241479f1d266a19359742 P bf3fc5736a524c77b298cb55de1d4d96 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:52:58.105914 25361 ts_tablet_manager.cc:1905] T 81699f3c6f0241479f1d266a19359742 P c8616d3510ea40428e98a492974ed0b0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:52:58.106386 25362 ts_tablet_manager.cc:1905] T 81699f3c6f0241479f1d266a19359742 P dec94c487df54d0fa23139ebda3881d4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:52:58.108212 25362 ts_tablet_manager.cc:1918] T 81699f3c6f0241479f1d266a19359742 P dec94c487df54d0fa23139ebda3881d4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 16:52:58.108282 25362 log.cc:1196] T 81699f3c6f0241479f1d266a19359742 P dec94c487df54d0fa23139ebda3881d4: Deleting WAL directory at /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786207919384338-8288-0/minicluster-data/ts-1/wal/wals/81699f3c6f0241479f1d266a19359742
I20260808 16:52:58.108558 25362 ts_tablet_manager.cc:1939] T 81699f3c6f0241479f1d266a19359742 P dec94c487df54d0fa23139ebda3881d4: Deleting consensus metadata
I20260808 16:52:58.108911 24574 catalog_manager.cc:4841] TS dec94c487df54d0fa23139ebda3881d4 (127.8.24.2:46797): tablet 81699f3c6f0241479f1d266a19359742 (table test-table [id=3488af28dc1243559feec270182e03d0]) successfully deleted
I20260808 16:52:58.109558 25366 ts_tablet_manager.cc:1905] T 81699f3c6f0241479f1d266a19359742 P bf3fc5736a524c77b298cb55de1d4d96: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:52:58.111177 25366 ts_tablet_manager.cc:1918] T 81699f3c6f0241479f1d266a19359742 P bf3fc5736a524c77b298cb55de1d4d96: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 16:52:58.111253 25366 log.cc:1196] T 81699f3c6f0241479f1d266a19359742 P bf3fc5736a524c77b298cb55de1d4d96: Deleting WAL directory at /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786207919384338-8288-0/minicluster-data/ts-2/wal/wals/81699f3c6f0241479f1d266a19359742
I20260808 16:52:58.111536 25366 ts_tablet_manager.cc:1939] T 81699f3c6f0241479f1d266a19359742 P bf3fc5736a524c77b298cb55de1d4d96: Deleting consensus metadata
I20260808 16:52:58.111945 24575 catalog_manager.cc:4841] TS bf3fc5736a524c77b298cb55de1d4d96 (127.8.24.3:37939): tablet 81699f3c6f0241479f1d266a19359742 (table test-table [id=3488af28dc1243559feec270182e03d0]) successfully deleted
W20260808 16:52:58.113694 24575 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.24.1:35569 (error 108)
W20260808 16:52:58.114719 25321 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:52:58.114023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:35569 (local address 127.0.0.1:37306)
0808 16:52:58.114073 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0808 16:52:58.114084 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 16:52:58.114123 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:52:58.114675 (+ 552us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.1:35569: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":32}
W20260808 16:52:58.114840 24575 catalog_manager.cc:4584] TS c8616d3510ea40428e98a492974ed0b0 (127.8.24.1:35569): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 81699f3c6f0241479f1d266a19359742: Network error: Client connection negotiation failed: client connection to 127.8.24.1:35569: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 16:52:58.115129 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 24889
I20260808 16:52:58.132491 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 25112
I20260808 16:52:58.141193 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 24550
2026-08-08T16:52:58Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2173 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 16:52:58 dist-test-slave-6zbq krb5kdc[25375](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 16:52:58 dist-test-slave-6zbq krb5kdc[25375](info): set up 2 sockets
Aug 08 16:52:58 dist-test-slave-6zbq krb5kdc[25375](info): commencing operation
krb5kdc: starting...
W20260808 16:52:59.263562 8288 mini_kdc.cc:121] Time spent starting KDC: real 1.100s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/krb5kdc/test-user.keytab.
Aug 08 16:52:59 dist-test-slave-6zbq krb5kdc[25375](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786207979, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T16:52:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T16:52:59Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.8.24.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.8.24.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.8.24.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/krb5kdc/rangeradmin_127.8.24.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.8.24.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/krb5kdc/rangeradmin_127.8.24.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.8.24.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.8.24.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.8.24.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/krb5kdc/rangerlookup_127.8.24.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.8.24.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/krb5kdc/rangerlookup_127.8.24.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.8.24.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.8.24.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.8.24.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/krb5kdc/HTTP_127.8.24.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.8.24.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/krb5kdc/HTTP_127.8.24.20@KRBTEST.COM.keytab.
I20260808 16:52:59.403038 8288 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskJykUMM/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/minicluster-data/postgres -l logfile start
2026-08-08 16:53:01.500 UTC [25420] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-08 16:53:01.500 UTC [25420] LOG: listening on IPv4 address "127.8.24.20", port 40497
2026-08-08 16:53:01.503 UTC [25420] LOG: listening on Unix socket "/tmp/.s.PGSQL.40497"
2026-08-08 16:53:01.529 UTC [25424] LOG: database system was shut down at 2026-08-08 16:53:00 UTC
2026-08-08 16:53:01.532 UTC [25420] LOG: database system is ready to accept connections
I20260808 16:53:02.584277 8288 mini_postgres.cc:96] Postgres bound to 40497
2026-08-08 16:53:02.592 UTC [25437] FATAL: database "slave" does not exist
127.8.24.20:40497 - accepting connections
I20260808 16:53:02.593269 8288 mini_ranger.cc:161] Starting Ranger...
I20260808 16:53:02.609156 8288 mini_ranger.cc:84] Created miniranger Postgres user
I20260808 16:53:02.986508 8288 mini_ranger.cc:87] Created ranger Postgres database
I20260808 16:53:02.986580 8288 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/minicluster-data/ranger-admin
2026-08-08 16:53:03,326 [I] DB FLAVOR :POSTGRES
2026-08-08 16:53:03,326 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 16:53:03,326 [I] Checking connection..
2026-08-08 16:53:03,326 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJykUMM/build/release/bin/postgresql.jar:/tmp/dist-test-taskJykUMM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.24.20:40497/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-08 16:53:03,587 [I] Checking connection passed.
2026-08-08 16:53:03,587 [I] --------- Verifying version history table ---------
2026-08-08 16:53:03,587 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJykUMM/build/release/bin/postgresql.jar:/tmp/dist-test-taskJykUMM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.24.20:40497/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 16:53:03,819 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 16:53:03,820 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJykUMM/build/release/bin/postgresql.jar:/tmp/dist-test-taskJykUMM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.24.20:40497/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 16:53:04,050 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 16:53:04,051 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-08 16:53:04,051 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJykUMM/build/release/bin/postgresql.jar:/tmp/dist-test-taskJykUMM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.24.20:40497/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskJykUMM/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-08 16:53:04.280 UTC [25538] WARNING: there is no transaction in progress
2026-08-08 16:53:04,285 [I] create_dbversion_catalog.sql file imported successfully
2026-08-08 16:53:04,286 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJykUMM/build/release/bin/postgresql.jar:/tmp/dist-test-taskJykUMM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.24.20:40497/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 16:53:04,517 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-08 16:53:04,517 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 16:53:04,517 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJykUMM/build/release/bin/postgresql.jar:/tmp/dist-test-taskJykUMM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.24.20:40497/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-08 16:53:04,764 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJykUMM/build/release/bin/postgresql.jar:/tmp/dist-test-taskJykUMM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.24.20:40497/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-08 16:53:05,001 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJykUMM/build/release/bin/postgresql.jar:/tmp/dist-test-taskJykUMM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.24.20:40497/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-6zbq.c.gcp-upstream.internal','N') ;"
2026-08-08 16:53:05,237 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJykUMM/build/release/bin/postgresql.jar:/tmp/dist-test-taskJykUMM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.24.20:40497/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 16:53:05,473 [I] Table x_portal_user does not exist in database ranger
2026-08-08 16:53:05,473 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJykUMM/build/release/bin/postgresql.jar:/tmp/dist-test-taskJykUMM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.24.20:40497/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 16:53:05,693 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-08 16:53:05,693 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJykUMM/build/release/bin/postgresql.jar:/tmp/dist-test-taskJykUMM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.24.20:40497/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 16:53:05,925 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJykUMM/build/release/bin/postgresql.jar:/tmp/dist-test-taskJykUMM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.24.20:40497/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 16:53:06,160 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-08 16:53:06,161 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJykUMM/build/release/bin/postgresql.jar:/tmp/dist-test-taskJykUMM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.24.20:40497/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskJykUMM/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-08 16:53:07.025 UTC [25697] WARNING: there is no transaction in progress
2026-08-08 16:53:07.039 UTC [25697] WARNING: there is no transaction in progress
2026-08-08 16:53:07.053 UTC [25697] WARNING: there is no transaction in progress
2026-08-08 16:53:07.066 UTC [25697] WARNING: there is no transaction in progress
2026-08-08 16:53:07.079 UTC [25697] WARNING: there is no transaction in progress
2026-08-08 16:53:07.095 UTC [25697] WARNING: there is no transaction in progress
2026-08-08 16:53:07.144 UTC [25697] WARNING: there is no transaction in progress
2026-08-08 16:53:07.150 UTC [25697] WARNING: there is no transaction in progress
2026-08-08 16:53:07.162 UTC [25697] WARNING: there is no transaction in progress
2026-08-08 16:53:07.171 UTC [25697] WARNING: there is no transaction in progress
2026-08-08 16:53:07.180 UTC [25697] WARNING: there is no transaction in progress
2026-08-08 16:53:07.191 UTC [25697] WARNING: there is no transaction in progress
2026-08-08 16:53:07.199 UTC [25697] WARNING: there is no transaction in progress
2026-08-08 16:53:07.204 UTC [25697] WARNING: there is no transaction in progress
2026-08-08 16:53:07.209 UTC [25697] WARNING: there is no transaction in progress
2026-08-08 16:53:07.214 UTC [25697] WARNING: there is no transaction in progress
2026-08-08 16:53:07,730 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-08 16:53:07,731 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJykUMM/build/release/bin/postgresql.jar:/tmp/dist-test-taskJykUMM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.24.20:40497/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-08 16:53:07,973 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-08 16:53:07,973 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJykUMM/build/release/bin/postgresql.jar:/tmp/dist-test-taskJykUMM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.24.20:40497/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 16:53:08,224 [I] Table x_portal_user already exists in database 'ranger'
2026-08-08 16:53:08,224 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJykUMM/build/release/bin/postgresql.jar:/tmp/dist-test-taskJykUMM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.24.20:40497/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 16:53:08,466 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-08 16:53:08,466 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJykUMM/build/release/bin/postgresql.jar:/tmp/dist-test-taskJykUMM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.24.20:40497/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 16:53:08,687 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJykUMM/build/release/bin/postgresql.jar:/tmp/dist-test-taskJykUMM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.24.20:40497/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 16:53:08,934 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJykUMM/build/release/bin/postgresql.jar:/tmp/dist-test-taskJykUMM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.24.20:40497/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-6zbq.c.gcp-upstream.internal';"
2026-08-08 16:53:09,157 [I] CORE_DB_SCHEMA import status has been updated
2026-08-08 16:53:09,157 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJykUMM/build/release/bin/postgresql.jar:/tmp/dist-test-taskJykUMM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.24.20:40497/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-08 16:53:09,387 [I] DB_PATCHES have already been applied
I20260808 16:53:09.392498 8288 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/minicluster-data/ranger-admin:/tmp/dist-test-taskJykUMM/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskJykUMM/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskJykUMM/build/release/bin/postgresql.jar:/tmp/dist-test-taskJykUMM/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260808 16:53:09.392557 8288 mini_ranger.cc:193] Using host: 127.8.24.20
I20260808 16:53:09.396349 8288 mini_ranger.cc:227] Ranger admin URL: http://127.8.24.20:42977
Aug 08, 2026 4:53:09 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 4:53:09 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 4:53:09 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/minicluster-data/ranger-admin/ews/webapp
Aug 08, 2026 4:53:09 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 08, 2026 4:53:09 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 08, 2026 4:53:10 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/minicluster-data/ranger-admin/ews/webapp].
Aug 08, 2026 4:53:10 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260808 16:53:10.629438 8288 mini_ranger.cc:160] Time spent starting Ranger: real 8.036s user 0.002s sys 0.008s
Aug 08, 2026 4:53:28 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 08, 2026 4:53:28 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 08, 2026 4:53:28 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 08, 2026 4:53:28 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 08, 2026 4:53:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 08, 2026 4:53:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 08, 2026 4:53:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 08, 2026 4:53:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 08, 2026 4:53:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 08, 2026 4:53:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 08, 2026 4:53:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 08, 2026 4:53:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 08, 2026 4:53:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 08, 2026 4:53:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 08, 2026 4:53:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 08, 2026 4:53:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 08, 2026 4:53:28 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 08, 2026 4:53:29 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260808 16:53:30.351634 8288 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.8.24.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.8.24.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.8.24.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/krb5kdc/rangerkms_127.8.24.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.8.24.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/krb5kdc/rangerkms_127.8.24.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.8.24.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.8.24.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.8.24.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/krb5kdc/HTTP_127.8.24.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.8.24.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/krb5kdc/HTTP_127.8.24.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 08 16:53:30 dist-test-slave-6zbq krb5kdc[25375](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208010, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260808 16:53:30.436419 8288 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260808 16:53:30.445330 8288 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260808 16:53:31.096854 8288 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260808 16:53:31.096937 8288 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/minicluster-data/ranger-kms
I20260808 16:53:31.096952 8288 mini_ranger_kms.cc:227] Using postgres at 127.8.24.20:40497
2026-08-08 16:53:31,304 [I] DB FLAVOR :POSTGRES
2026-08-08 16:53:31,304 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 16:53:31,304 [I] Checking connection
2026-08-08 16:53:31,305 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJykUMM/build/release/bin/postgresql.jar:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.24.20:40497/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-08 16:53:31,526 [I] connection success
2026-08-08 16:53:31,526 [I] --------- Verifying Ranger DB tables ---------
2026-08-08 16:53:31,526 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-08 16:53:31,527 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJykUMM/build/release/bin/postgresql.jar:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.24.20:40497/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-08 16:53:31,762 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-08 16:53:31,762 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 16:53:31,762 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-08 16:53:31,762 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJykUMM/build/release/bin/postgresql.jar:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.24.20:40497/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-08 16:53:32,012 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260808 16:53:32.448253 8288 mini_ranger_kms.cc:310] Created kms service
I20260808 16:53:32.684216 8288 mini_ranger_kms.cc:326] Created kudu user
I20260808 16:53:32.725473 8288 mini_ranger_kms.cc:343] Created rangerkms user
Aug 08 16:53:32 dist-test-slave-6zbq krb5kdc[25375](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208010, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.8.24.20@KRBTEST.COM
I20260808 16:53:33.115154 8288 mini_ranger_kms.cc:384] Added ranger policy
I20260808 16:53:33.115260 8288 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/minicluster-data/ranger-kms:/tmp/dist-test-taskJykUMM/build/release/bin/postgresql.jar:/tmp/dist-test-taskJykUMM/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskJykUMM/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260808 16:53:33.115278 8288 mini_ranger_kms.cc:240] Using host: 127.8.24.20
I20260808 16:53:33.119676 8288 mini_ranger_kms.cc:276] Ranger KMS PID: 25995
I20260808 16:53:33.119773 8288 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.8.24.20:34481
Aug 08, 2026 4:53:34 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 4:53:34 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 4:53:34 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 08, 2026 4:53:34 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 08, 2026 4:53:34 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/minicluster-data/ranger-kms/ews/webapp].
Aug 08, 2026 4:53:34 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 08 16:53:35 dist-test-slave-6zbq krb5kdc[25375](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786208015, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.24.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:53:35.616850 8288 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.180s user 0.008s sys 0.183s
I20260808 16:53:35.616966 8288 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260808 16:53:35.617050 8288 mini_ranger_kms.cc:401] 127.8.24.20:34481/kms/v1/keys
Aug 08 16:53:35 dist-test-slave-6zbq krb5kdc[25375](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208015, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.24.20@KRBTEST.COM for HTTP/127.8.24.20@KRBTEST.COM
Aug 08 16:53:36 dist-test-slave-6zbq krb5kdc[25375](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208015, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.24.20@KRBTEST.COM for HTTP/127.8.24.20@KRBTEST.COM
Aug 08 16:53:38 dist-test-slave-6zbq krb5kdc[25375](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208018, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.8.24.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
I20260808 16:53:38.336500 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:40269
--webserver_interface=127.8.24.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:36621
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:40269
--ranger_config_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.24.20:34481/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:53:38.423192 26066 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:53:38.423368 26066 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:53:38.423388 26066 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:53:38.424500 26066 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 16:53:38.424539 26066 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:53:38.424551 26066 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:53:38.424562 26066 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 16:53:38.424574 26066 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 16:53:38.426126 26066 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:36621
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/minicluster-data/master-0/wal
--ranger_kms_url=127.8.24.20:34481/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:40269
--ranger_config_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:40269
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.24.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26066
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:53:38.426355 26066 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:53:38.426602 26066 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:53:38.431362 26071 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:53:38.431460 26077 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:53:38.431476 26066 server_base.cc:1034] running on GCE node
W20260808 16:53:38.431363 26074 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:53:38.431764 26066 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:53:38.432042 26066 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:53:38.433197 26066 hybrid_clock.cc:648] HybridClock initialized: now 1786208018433180 us; error 34 us; skew 500 ppm
Aug 08 16:53:38 dist-test-slave-6zbq krb5kdc[25375](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208018, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:53:38.440049 26066 init.cc:377] Logged in from keytab as kudu/127.8.24.62@KRBTEST.COM (short username kudu)
I20260808 16:53:38.440557 26066 webserver.cc:460] Webserver started at http://127.8.24.62:42197/ using document root <none> and password file <none>
I20260808 16:53:38.440778 26066 fs_manager.cc:359] Metadata directory not provided
I20260808 16:53:38.440827 26066 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:53:38.440950 26066 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 16:53:38 dist-test-slave-6zbq krb5kdc[25375](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208018, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for HTTP/127.8.24.20@KRBTEST.COM
I20260808 16:53:38.485698 26066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/minicluster-data/master-0/data/instance:
uuid: "30126cedc2a64c66b655b04a7d623041"
format_stamp: "Formatted at 2026-08-08 16:53:38 on dist-test-slave-6zbq"
server_key: "af44a1f422e3abab908ef63e8c39ebed"
server_key_iv: "8127ddcfd6168c4466c96134a0a56cfb"
server_key_version: "kuduclusterkey@0"
I20260808 16:53:38.486138 26066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/minicluster-data/master-0/wal/instance:
uuid: "30126cedc2a64c66b655b04a7d623041"
format_stamp: "Formatted at 2026-08-08 16:53:38 on dist-test-slave-6zbq"
server_key: "af44a1f422e3abab908ef63e8c39ebed"
server_key_iv: "8127ddcfd6168c4466c96134a0a56cfb"
server_key_version: "kuduclusterkey@0"
I20260808 16:53:38.488147 26066 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.001s
I20260808 16:53:38.498889 26087 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:53:38.499059 26066 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 16:53:38.499164 26066 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/minicluster-data/master-0/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/minicluster-data/master-0/wal
uuid: "30126cedc2a64c66b655b04a7d623041"
format_stamp: "Formatted at 2026-08-08 16:53:38 on dist-test-slave-6zbq"
server_key: "af44a1f422e3abab908ef63e8c39ebed"
server_key_iv: "8127ddcfd6168c4466c96134a0a56cfb"
server_key_version: "kuduclusterkey@0"
I20260808 16:53:38.499245 26066 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:53:38.520879 26066 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:53:38.522714 26066 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:53:38.522840 26066 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:53:38.527374 26066 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.62:40269
I20260808 16:53:38.527400 26182 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.62:40269 every 8 connection(s)
I20260808 16:53:38.527725 26066 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
I20260808 16:53:38.528358 26183 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:53:38.530553 26183 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30126cedc2a64c66b655b04a7d623041: Bootstrap starting.
I20260808 16:53:38.531131 26183 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 30126cedc2a64c66b655b04a7d623041: Neither blocks nor log segments found. Creating new log.
I20260808 16:53:38.531369 26183 log.cc:824] T 00000000000000000000000000000000 P 30126cedc2a64c66b655b04a7d623041: Log is configured to *not* fsync() on all Append() calls
I20260808 16:53:38.531704 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 26066
I20260808 16:53:38.531806 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/minicluster-data/master-0/wal/instance
I20260808 16:53:38.531977 8288 external_mini_cluster.cc:1428] Setting key 856e8bde08c98181baa4dc14a613c1c7
I20260808 16:53:38.532021 26183 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30126cedc2a64c66b655b04a7d623041: No bootstrap required, opened a new log
I20260808 16:53:38.534756 26183 raft_consensus.cc:348] T 00000000000000000000000000000000 P 30126cedc2a64c66b655b04a7d623041 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30126cedc2a64c66b655b04a7d623041" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 40269 } }
I20260808 16:53:38.534888 26183 raft_consensus.cc:374] T 00000000000000000000000000000000 P 30126cedc2a64c66b655b04a7d623041 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:53:38.534919 26183 raft_consensus.cc:729] T 00000000000000000000000000000000 P 30126cedc2a64c66b655b04a7d623041 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30126cedc2a64c66b655b04a7d623041, State: Initialized, Role: FOLLOWER
I20260808 16:53:38.535070 26183 consensus_queue.cc:260] T 00000000000000000000000000000000 P 30126cedc2a64c66b655b04a7d623041 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30126cedc2a64c66b655b04a7d623041" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 40269 } }
I20260808 16:53:38.535137 26183 raft_consensus.cc:388] T 00000000000000000000000000000000 P 30126cedc2a64c66b655b04a7d623041 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 16:53:38.535167 26183 raft_consensus.cc:482] T 00000000000000000000000000000000 P 30126cedc2a64c66b655b04a7d623041 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 16:53:38.535228 26183 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 30126cedc2a64c66b655b04a7d623041 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:53:38.535712 26183 raft_consensus.cc:504] T 00000000000000000000000000000000 P 30126cedc2a64c66b655b04a7d623041 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30126cedc2a64c66b655b04a7d623041" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 40269 } }
I20260808 16:53:38.535805 26183 leader_election.cc:302] T 00000000000000000000000000000000 P 30126cedc2a64c66b655b04a7d623041 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 30126cedc2a64c66b655b04a7d623041; no voters:
I20260808 16:53:38.535991 26183 leader_election.cc:288] T 00000000000000000000000000000000 P 30126cedc2a64c66b655b04a7d623041 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 16:53:38.536031 26192 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 30126cedc2a64c66b655b04a7d623041 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:53:38.536221 26192 raft_consensus.cc:686] T 00000000000000000000000000000000 P 30126cedc2a64c66b655b04a7d623041 [term 1 LEADER]: Becoming Leader. State: Replica: 30126cedc2a64c66b655b04a7d623041, State: Running, Role: LEADER
I20260808 16:53:38.536262 26183 sys_catalog.cc:564] T 00000000000000000000000000000000 P 30126cedc2a64c66b655b04a7d623041 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 16:53:38.536417 26192 consensus_queue.cc:237] T 00000000000000000000000000000000 P 30126cedc2a64c66b655b04a7d623041 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30126cedc2a64c66b655b04a7d623041" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 40269 } }
I20260808 16:53:38.539531 26183 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
Aug 08 16:53:38 dist-test-slave-6zbq krb5kdc[25375](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208018, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260808 16:53:38.539001 26194 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30126cedc2a64c66b655b04a7d623041 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 30126cedc2a64c66b655b04a7d623041. Latest consensus state: current_term: 1 leader_uuid: "30126cedc2a64c66b655b04a7d623041" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30126cedc2a64c66b655b04a7d623041" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 40269 } } }
I20260808 16:53:38.540604 26194 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30126cedc2a64c66b655b04a7d623041 [sys.catalog]: This master's current role is: LEADER
I20260808 16:53:38.543253 26193 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30126cedc2a64c66b655b04a7d623041 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "30126cedc2a64c66b655b04a7d623041" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30126cedc2a64c66b655b04a7d623041" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 40269 } } }
I20260808 16:53:38.543469 26193 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30126cedc2a64c66b655b04a7d623041 [sys.catalog]: This master's current role is: LEADER
I20260808 16:53:38.545768 26219 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 16:53:38.546437 26219 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 16:53:38.546747 26190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:53:38.532506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48302 (local address 127.8.24.62:40269)
0808 16:53:38.532689 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0808 16:53:38.532698 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 16:53:38.532727 (+ 29us) server_negotiation.cc:407] Connection header received
0808 16:53:38.533419 (+ 692us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:53:38.533429 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:53:38.533497 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:53:38.533651 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:53:38.534405 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:53:38.535783 (+ 1378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:53:38.537645 (+ 1862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:53:38.538254 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:53:38.538388 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:53:38.542620 (+ 4232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:53:38.542666 (+ 46us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:53:38.542666 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:53:38.542746 (+ 80us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:53:38.545066 (+ 2320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:53:38.545429 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:53:38.545432 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:53:38.545433 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:53:38.545509 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:53:38.545699 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:53:38.545700 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:53:38.545700 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:53:38.546016 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:53:38.546156 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0808 16:53:38.546266 (+ 110us) server_negotiation.cc:299] Negotiation successful
0808 16:53:38.546533 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":92,"threads_started":1}
I20260808 16:53:38.548215 26219 catalog_manager.cc:1347] Generated new cluster ID: b66c136bfba04842a7d0b648cf567b14
I20260808 16:53:38.548275 26219 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 16:53:38.570262 26219 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 16:53:38.571039 26219 catalog_manager.cc:1504] Loading token signing keys...
I20260808 16:53:38.581333 26219 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 30126cedc2a64c66b655b04a7d623041: Generated new TSK 0
I20260808 16:53:38.581557 26219 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.24.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.1.keytab.
Entry for principal kudu/127.8.24.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.1.keytab.
I20260808 16:53:39.228914 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.24.20:34481/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.1:0
--local_ip_for_outbound_sockets=127.8.24.1
--webserver_interface=127.8.24.1
--webserver_port=0
--tserver_master_addrs=127.8.24.62:40269
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.1.keytab
--principal=kudu/127.8.24.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:36621
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:53:39.344416 26257 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:53:39.344667 26257 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:53:39.344699 26257 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:53:39.346297 26257 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:53:39.346365 26257 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:53:39.346415 26257 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.1
I20260808 16:53:39.348464 26257 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:36621
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.8.24.20:34481/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.1.keytab
--principal=kudu/127.8.24.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.24.1
--webserver_port=0
--tserver_master_addrs=127.8.24.62:40269
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26257
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.1
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:53:39.348699 26257 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:53:39.348942 26257 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:53:39.353906 26264 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:53:39.353917 26273 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:53:39.354111 26257 server_base.cc:1034] running on GCE node
W20260808 16:53:39.354132 26263 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:53:39.354423 26257 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:53:39.354702 26257 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:53:39.355878 26257 hybrid_clock.cc:648] HybridClock initialized: now 1786208019355860 us; error 31 us; skew 500 ppm
Aug 08 16:53:39 dist-test-slave-6zbq krb5kdc[25375](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208019, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:53:39.358635 26257 init.cc:377] Logged in from keytab as kudu/127.8.24.1@KRBTEST.COM (short username kudu)
I20260808 16:53:39.359045 26257 webserver.cc:460] Webserver started at http://127.8.24.1:46461/ using document root <none> and password file <none>
I20260808 16:53:39.359270 26257 fs_manager.cc:359] Metadata directory not provided
I20260808 16:53:39.359318 26257 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:53:39.359467 26257 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 16:53:39 dist-test-slave-6zbq krb5kdc[25375](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208019, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.1@KRBTEST.COM for HTTP/127.8.24.20@KRBTEST.COM
I20260808 16:53:39.371382 26257 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/minicluster-data/ts-0/data/instance:
uuid: "d9f52b7084da4804b09a44ec028fb18d"
format_stamp: "Formatted at 2026-08-08 16:53:39 on dist-test-slave-6zbq"
server_key: "3fe820d1a6317554778713a9fb651e0b"
server_key_iv: "af2fad6926f917286c9284c2848ce0e7"
server_key_version: "kuduclusterkey@0"
I20260808 16:53:39.371704 26257 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/minicluster-data/ts-0/wal/instance:
uuid: "d9f52b7084da4804b09a44ec028fb18d"
format_stamp: "Formatted at 2026-08-08 16:53:39 on dist-test-slave-6zbq"
server_key: "3fe820d1a6317554778713a9fb651e0b"
server_key_iv: "af2fad6926f917286c9284c2848ce0e7"
server_key_version: "kuduclusterkey@0"
I20260808 16:53:39.373064 26257 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260808 16:53:39.386364 26286 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:53:39.386525 26257 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 16:53:39.386626 26257 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/minicluster-data/ts-0/wal
uuid: "d9f52b7084da4804b09a44ec028fb18d"
format_stamp: "Formatted at 2026-08-08 16:53:39 on dist-test-slave-6zbq"
server_key: "3fe820d1a6317554778713a9fb651e0b"
server_key_iv: "af2fad6926f917286c9284c2848ce0e7"
server_key_version: "kuduclusterkey@0"
I20260808 16:53:39.386718 26257 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:53:39.402995 26257 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:53:39.412567 26257 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:53:39.412747 26257 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:53:39.412940 26257 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:53:39.413251 26257 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:53:39.413291 26257 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:53:39.413319 26257 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:53:39.413357 26257 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:53:39.419368 26257 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.1:43351
I20260808 16:53:39.420163 26257 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/minicluster-data/ts-0/data/info.pb
I20260808 16:53:39.421461 26474 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.1:43351 every 8 connection(s)
I20260808 16:53:39.424911 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 26257
I20260808 16:53:39.425002 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/minicluster-data/ts-0/wal/instance
I20260808 16:53:39.425199 8288 external_mini_cluster.cc:1428] Setting key 15c20afb8c1b5f7e5dad3983d14f3421
Aug 08 16:53:39 dist-test-slave-6zbq krb5kdc[25375](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208019, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.1@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260808 16:53:39.432658 26478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:53:39.421192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:40269 (local address 127.8.24.1:42595)
0808 16:53:39.422468 (+ 1276us) negotiation.cc:107] Waiting for socket to connect
0808 16:53:39.422508 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 16:53:39.423101 (+ 593us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:53:39.423516 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:53:39.423521 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:53:39.423752 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:53:39.424016 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:53:39.424023 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:53:39.425012 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:53:39.425013 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:53:39.426227 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:53:39.426230 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:53:39.426796 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:53:39.426797 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:53:39.426940 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:53:39.427471 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:53:39.427481 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:53:39.429108 (+ 1627us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:53:39.431070 (+ 1962us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:53:39.431077 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:53:39.431079 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:53:39.431354 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:53:39.431586 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:53:39.431587 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:53:39.431587 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:53:39.431743 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:53:39.432039 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:53:39.432045 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:53:39.432205 (+ 160us) client_negotiation.cc:769] Sending connection context
0808 16:53:39.432309 (+ 104us) client_negotiation.cc:240] Negotiation successful
0808 16:53:39.432447 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1235,"thread_start_us":95,"threads_started":1}
I20260808 16:53:39.433089 26479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:53:39.421220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:42595 (local address 127.8.24.62:40269)
0808 16:53:39.423291 (+ 2071us) server_negotiation.cc:206] Beginning negotiation
0808 16:53:39.423293 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:53:39.423301 (+ 8us) server_negotiation.cc:407] Connection header received
0808 16:53:39.423325 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:53:39.423326 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:53:39.423337 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:53:39.423394 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:53:39.424128 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:53:39.424901 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:53:39.426320 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:53:39.426650 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:53:39.426728 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:53:39.429213 (+ 2485us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:53:39.429214 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:53:39.429215 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:53:39.429222 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:53:39.430975 (+ 1753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:53:39.431441 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:53:39.431443 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:53:39.431443 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:53:39.431512 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:53:39.431827 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:53:39.431828 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:53:39.431828 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:53:39.431961 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:53:39.432015 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 16:53:39.432904 (+ 889us) server_negotiation.cc:299] Negotiation successful
0808 16:53:39.432987 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2044,"thread_start_us":81,"threads_started":1}
I20260808 16:53:39.433540 26475 heartbeater.cc:347] Connected to a master server at 127.8.24.62:40269
I20260808 16:53:39.433627 26475 heartbeater.cc:464] Registering TS with master...
I20260808 16:53:39.433789 26475 heartbeater.cc:511] Master 127.8.24.62:40269 requested a full tablet report, sending...
I20260808 16:53:39.434551 26119 ts_manager.cc:194] Registered new tserver with Master: d9f52b7084da4804b09a44ec028fb18d (127.8.24.1:43351)
I20260808 16:53:39.435407 26119 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.1@KRBTEST.COM'} at 127.8.24.1:42595
WARNING: no policy specified for kudu/127.8.24.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.2.keytab.
Entry for principal kudu/127.8.24.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.2.keytab.
I20260808 16:53:39.463742 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.24.20:34481/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.2:0
--local_ip_for_outbound_sockets=127.8.24.2
--webserver_interface=127.8.24.2
--webserver_port=0
--tserver_master_addrs=127.8.24.62:40269
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.2.keytab
--principal=kudu/127.8.24.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:36621
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:53:39.582497 26484 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:53:39.582819 26484 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:53:39.582907 26484 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:53:39.584383 26484 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:53:39.584479 26484 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:53:39.584554 26484 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.2
I20260808 16:53:39.586669 26484 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:36621
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.8.24.20:34481/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.2.keytab
--principal=kudu/127.8.24.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.24.2
--webserver_port=0
--tserver_master_addrs=127.8.24.62:40269
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26484
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.2
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:53:39.587100 26484 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:53:39.587378 26484 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:53:39.593721 26501 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:53:39.594683 26484 server_base.cc:1034] running on GCE node
W20260808 16:53:39.596992 26496 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:53:39.597206 26495 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:53:39.597465 26484 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:53:39.597702 26484 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:53:39.598838 26484 hybrid_clock.cc:648] HybridClock initialized: now 1786208019598819 us; error 32 us; skew 500 ppm
Aug 08 16:53:39 dist-test-slave-6zbq krb5kdc[25375](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208019, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:53:39.601833 26484 init.cc:377] Logged in from keytab as kudu/127.8.24.2@KRBTEST.COM (short username kudu)
I20260808 16:53:39.602317 26484 webserver.cc:460] Webserver started at http://127.8.24.2:41855/ using document root <none> and password file <none>
I20260808 16:53:39.602555 26484 fs_manager.cc:359] Metadata directory not provided
I20260808 16:53:39.602600 26484 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:53:39.602721 26484 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 16:53:39 dist-test-slave-6zbq krb5kdc[25375](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208019, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.2@KRBTEST.COM for HTTP/127.8.24.20@KRBTEST.COM
I20260808 16:53:39.617084 26484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/minicluster-data/ts-1/data/instance:
uuid: "25de29a6a6924edfafa087050138b7b4"
format_stamp: "Formatted at 2026-08-08 16:53:39 on dist-test-slave-6zbq"
server_key: "1a8ed388ab5dbded80d13a8247fd97c5"
server_key_iv: "0472a5d8ec30f47ea5a8cac28a4eebcf"
server_key_version: "kuduclusterkey@0"
I20260808 16:53:39.617728 26484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/minicluster-data/ts-1/wal/instance:
uuid: "25de29a6a6924edfafa087050138b7b4"
format_stamp: "Formatted at 2026-08-08 16:53:39 on dist-test-slave-6zbq"
server_key: "1a8ed388ab5dbded80d13a8247fd97c5"
server_key_iv: "0472a5d8ec30f47ea5a8cac28a4eebcf"
server_key_version: "kuduclusterkey@0"
I20260808 16:53:39.619936 26484 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260808 16:53:39.639513 26509 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:53:39.641469 26484 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.000s sys 0.000s
I20260808 16:53:39.641572 26484 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/minicluster-data/ts-1/wal
uuid: "25de29a6a6924edfafa087050138b7b4"
format_stamp: "Formatted at 2026-08-08 16:53:39 on dist-test-slave-6zbq"
server_key: "1a8ed388ab5dbded80d13a8247fd97c5"
server_key_iv: "0472a5d8ec30f47ea5a8cac28a4eebcf"
server_key_version: "kuduclusterkey@0"
I20260808 16:53:39.641674 26484 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
Aug 08 16:53:39 dist-test-slave-6zbq krb5kdc[25375](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786208019, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:53:39.668543 26484 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:53:39.670931 26484 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:53:39.671362 26484 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:53:39.671706 26484 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:53:39.672118 26484 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:53:39.672205 26484 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:53:39.672274 26484 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:53:39.672335 26484 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:53:39.678918 26484 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.2:42815
I20260808 16:53:39.679390 26484 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/minicluster-data/ts-1/data/info.pb
I20260808 16:53:39.681900 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 26484
I20260808 16:53:39.682006 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/minicluster-data/ts-1/wal/instance
I20260808 16:53:39.682209 8288 external_mini_cluster.cc:1428] Setting key 30a4f9a2817797c7aafb10a86dd7bdef
I20260808 16:53:39.682904 26678 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.2:42815 every 8 connection(s)
Aug 08 16:53:39 dist-test-slave-6zbq krb5kdc[25375](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208019, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.2@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.24.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.3@KRBTEST.COM" created.
I20260808 16:53:39.705907 26479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:53:39.685667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:48395 (local address 127.8.24.62:40269)
0808 16:53:39.685719 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 16:53:39.685721 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:53:39.690673 (+ 4952us) server_negotiation.cc:407] Connection header received
0808 16:53:39.691410 (+ 737us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:53:39.691412 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:53:39.691425 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:53:39.691455 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:53:39.694449 (+ 2994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:53:39.695209 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:53:39.696702 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:53:39.697004 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:53:39.697049 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:53:39.702330 (+ 5281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:53:39.702332 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:53:39.702333 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:53:39.702366 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:53:39.703984 (+ 1618us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:53:39.704631 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:53:39.704632 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:53:39.704633 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:53:39.704695 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:53:39.704964 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:53:39.704964 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:53:39.704965 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:53:39.705244 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:53:39.705302 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 16:53:39.705715 (+ 413us) server_negotiation.cc:299] Negotiation successful
0808 16:53:39.705809 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 16:53:39.707973 26683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:53:39.685392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:40269 (local address 127.8.24.2:48395)
0808 16:53:39.690442 (+ 5050us) negotiation.cc:107] Waiting for socket to connect
0808 16:53:39.690494 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 16:53:39.691278 (+ 784us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:53:39.691641 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:53:39.691646 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:53:39.691885 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:53:39.692177 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:53:39.692184 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:53:39.695333 (+ 3149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:53:39.695334 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:53:39.696593 (+ 1259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:53:39.696597 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:53:39.697166 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:53:39.697167 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:53:39.697315 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:53:39.699842 (+ 2527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:53:39.699853 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:53:39.702215 (+ 2362us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:53:39.704237 (+ 2022us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:53:39.704244 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:53:39.704246 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:53:39.704532 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:53:39.704761 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:53:39.704762 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:53:39.704762 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:53:39.704891 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:53:39.705337 (+ 446us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:53:39.705343 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:53:39.705516 (+ 173us) client_negotiation.cc:769] Sending connection context
0808 16:53:39.705627 (+ 111us) client_negotiation.cc:240] Negotiation successful
0808 16:53:39.707744 (+ 2117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4999,"thread_start_us":75,"threads_started":1}
I20260808 16:53:39.708576 26680 heartbeater.cc:347] Connected to a master server at 127.8.24.62:40269
I20260808 16:53:39.708673 26680 heartbeater.cc:464] Registering TS with master...
I20260808 16:53:39.708858 26680 heartbeater.cc:511] Master 127.8.24.62:40269 requested a full tablet report, sending...
I20260808 16:53:39.709249 26119 ts_manager.cc:194] Registered new tserver with Master: 25de29a6a6924edfafa087050138b7b4 (127.8.24.2:42815)
I20260808 16:53:39.709731 26119 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.2@KRBTEST.COM'} at 127.8.24.2:48395
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.3.keytab.
Entry for principal kudu/127.8.24.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.3.keytab.
I20260808 16:53:39.723505 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.24.20:34481/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.3:0
--local_ip_for_outbound_sockets=127.8.24.3
--webserver_interface=127.8.24.3
--webserver_port=0
--tserver_master_addrs=127.8.24.62:40269
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.3.keytab
--principal=kudu/127.8.24.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:36621
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:53:39.844041 26687 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:53:39.844383 26687 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:53:39.844460 26687 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:53:39.846128 26687 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:53:39.846238 26687 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:53:39.846326 26687 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.3
I20260808 16:53:39.848624 26687 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:36621
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.8.24.20:34481/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.3.keytab
--principal=kudu/127.8.24.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.24.3
--webserver_port=0
--tserver_master_addrs=127.8.24.62:40269
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26687
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.3
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:53:39.849071 26687 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:53:39.849376 26687 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:53:39.856055 26695 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:53:39.856400 26694 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:53:39.856643 26699 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:53:39.857399 26687 server_base.cc:1034] running on GCE node
I20260808 16:53:39.857703 26687 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:53:39.857954 26687 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:53:39.859112 26687 hybrid_clock.cc:648] HybridClock initialized: now 1786208019859089 us; error 37 us; skew 500 ppm
Aug 08 16:53:39 dist-test-slave-6zbq krb5kdc[25375](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208019, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:53:39.862440 26687 init.cc:377] Logged in from keytab as kudu/127.8.24.3@KRBTEST.COM (short username kudu)
I20260808 16:53:39.862919 26687 webserver.cc:460] Webserver started at http://127.8.24.3:34625/ using document root <none> and password file <none>
I20260808 16:53:39.863163 26687 fs_manager.cc:359] Metadata directory not provided
I20260808 16:53:39.863222 26687 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:53:39.863353 26687 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 16:53:39 dist-test-slave-6zbq krb5kdc[25375](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208019, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.3@KRBTEST.COM for HTTP/127.8.24.20@KRBTEST.COM
I20260808 16:53:39.885530 26687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/minicluster-data/ts-2/data/instance:
uuid: "41e9225a2ecb45d0969cd8c5d07bdff9"
format_stamp: "Formatted at 2026-08-08 16:53:39 on dist-test-slave-6zbq"
server_key: "034614ec6d184e0dab2068c140f265b6"
server_key_iv: "ea3bee9452b7e15f7907d8b34fa44f62"
server_key_version: "kuduclusterkey@0"
I20260808 16:53:39.886144 26687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/minicluster-data/ts-2/wal/instance:
uuid: "41e9225a2ecb45d0969cd8c5d07bdff9"
format_stamp: "Formatted at 2026-08-08 16:53:39 on dist-test-slave-6zbq"
server_key: "034614ec6d184e0dab2068c140f265b6"
server_key_iv: "ea3bee9452b7e15f7907d8b34fa44f62"
server_key_version: "kuduclusterkey@0"
I20260808 16:53:39.889465 26687 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.002s
I20260808 16:53:39.914250 26712 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:53:39.914491 26687 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.001s sys 0.000s
I20260808 16:53:39.914588 26687 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/minicluster-data/ts-2/wal
uuid: "41e9225a2ecb45d0969cd8c5d07bdff9"
format_stamp: "Formatted at 2026-08-08 16:53:39 on dist-test-slave-6zbq"
server_key: "034614ec6d184e0dab2068c140f265b6"
server_key_iv: "ea3bee9452b7e15f7907d8b34fa44f62"
server_key_version: "kuduclusterkey@0"
I20260808 16:53:39.914669 26687 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:53:39.923938 26687 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:53:39.925662 26687 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:53:39.925784 26687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:53:39.925982 26687 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:53:39.926297 26687 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:53:39.926357 26687 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:53:39.926388 26687 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:53:39.926416 26687 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:53:39.932528 26687 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.3:44031
I20260808 16:53:39.932622 26897 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.3:44031 every 8 connection(s)
I20260808 16:53:39.932955 26687 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/minicluster-data/ts-2/data/info.pb
Aug 08 16:53:39 dist-test-slave-6zbq krb5kdc[25375](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208019, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.3@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260808 16:53:39.942443 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 26687
I20260808 16:53:39.942541 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/minicluster-data/ts-2/wal/instance
I20260808 16:53:39.942719 8288 external_mini_cluster.cc:1428] Setting key 296c3ec647326427810a42eb6ad84f9c
I20260808 16:53:39.944615 26479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:53:39.934728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:45079 (local address 127.8.24.62:40269)
0808 16:53:39.934772 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 16:53:39.934774 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:53:39.934784 (+ 10us) server_negotiation.cc:407] Connection header received
0808 16:53:39.935293 (+ 509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:53:39.935295 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:53:39.935307 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:53:39.935355 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:53:39.935992 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:53:39.936762 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:53:39.938236 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:53:39.938561 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:53:39.938642 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:53:39.941027 (+ 2385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:53:39.941028 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:53:39.941028 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:53:39.941037 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:53:39.942684 (+ 1647us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:53:39.943547 (+ 863us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:53:39.943548 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:53:39.943548 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:53:39.943610 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:53:39.943915 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:53:39.943915 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:53:39.943916 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:53:39.944042 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:53:39.944126 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 16:53:39.944449 (+ 323us) server_negotiation.cc:299] Negotiation successful
0808 16:53:39.944528 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 16:53:39.944804 26902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:53:39.934478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:40269 (local address 127.8.24.3:45079)
0808 16:53:39.934637 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0808 16:53:39.934677 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 16:53:39.935193 (+ 516us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:53:39.935431 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:53:39.935434 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:53:39.935603 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:53:39.935877 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:53:39.935884 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:53:39.936878 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:53:39.936880 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:53:39.938149 (+ 1269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:53:39.938152 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:53:39.938651 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:53:39.938652 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:53:39.938807 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:53:39.939343 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:53:39.939353 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:53:39.940923 (+ 1570us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:53:39.942768 (+ 1845us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:53:39.942775 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:53:39.942777 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:53:39.943474 (+ 697us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:53:39.943692 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:53:39.943693 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:53:39.943693 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:53:39.943842 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:53:39.944112 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:53:39.944119 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:53:39.944301 (+ 182us) client_negotiation.cc:769] Sending connection context
0808 16:53:39.944425 (+ 124us) client_negotiation.cc:240] Negotiation successful
0808 16:53:39.944583 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":78,"threads_started":1}
I20260808 16:53:39.945868 26899 heartbeater.cc:347] Connected to a master server at 127.8.24.62:40269
I20260808 16:53:39.945950 26899 heartbeater.cc:464] Registering TS with master...
I20260808 16:53:39.946164 26899 heartbeater.cc:511] Master 127.8.24.62:40269 requested a full tablet report, sending...
I20260808 16:53:39.946579 26119 ts_manager.cc:194] Registered new tserver with Master: 41e9225a2ecb45d0969cd8c5d07bdff9 (127.8.24.3:44031)
I20260808 16:53:39.947067 26119 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.3@KRBTEST.COM'} at 127.8.24.3:45079
I20260808 16:53:39.954440 8288 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 16:53:39.966554 26479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:53:39.956097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48308 (local address 127.8.24.62:40269)
0808 16:53:39.956135 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0808 16:53:39.956136 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 16:53:39.956288 (+ 152us) server_negotiation.cc:407] Connection header received
0808 16:53:39.956377 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:53:39.956378 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:53:39.956387 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:53:39.956418 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:53:39.957115 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:53:39.957862 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:53:39.962759 (+ 4897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:53:39.963041 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:53:39.963110 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:53:39.964470 (+ 1360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:53:39.964471 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:53:39.964471 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:53:39.964478 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:53:39.965557 (+ 1079us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:53:39.965856 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:53:39.965857 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:53:39.965857 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:53:39.965905 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:53:39.966068 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:53:39.966069 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:53:39.966069 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:53:39.966165 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:53:39.966248 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 16:53:39.966397 (+ 149us) server_negotiation.cc:299] Negotiation successful
0808 16:53:39.966472 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 16:53:39.968086 26119 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48308:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 16:53:39.969622 26119 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 16:53:39.976280 26919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:53:39.972047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:43351 (local address 127.0.0.1:59434)
0808 16:53:39.972192 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0808 16:53:39.972229 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 16:53:39.972288 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:53:39.972936 (+ 648us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:53:39.972939 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:53:39.972942 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:53:39.973010 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:53:39.973013 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:53:39.973877 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:53:39.973879 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:53:39.975053 (+ 1174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:53:39.975056 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:53:39.975966 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:53:39.975967 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:53:39.976060 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:53:39.976073 (+ 13us) client_negotiation.cc:769] Sending connection context
0808 16:53:39.976138 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 16:53:39.976205 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":101,"threads_started":1}
I20260808 16:53:39.976280 26925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:53:39.972502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59434 (local address 127.8.24.1:43351)
0808 16:53:39.972713 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0808 16:53:39.972716 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 16:53:39.972726 (+ 10us) server_negotiation.cc:407] Connection header received
0808 16:53:39.972761 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:53:39.972763 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:53:39.972780 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:53:39.972923 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:53:39.973075 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:53:39.973738 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:53:39.975171 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:53:39.975887 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:53:39.975968 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:53:39.976027 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 16:53:39.976165 (+ 138us) server_negotiation.cc:299] Negotiation successful
0808 16:53:39.976213 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":64,"threads_started":1}
I20260808 16:53:39.976986 26369 tablet_service.cc:1467] Processing CreateTablet for tablet 05abe5347c614a6388289226ca7d2bc4 (DEFAULT_TABLE table=test-table [id=f4b1e19d808e4ebbaa7923ac3d20d92a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:53:39.977332 26369 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05abe5347c614a6388289226ca7d2bc4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:53:39.978031 26918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:53:39.972049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.2:42815 (local address 127.0.0.1:44862)
0808 16:53:39.972387 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0808 16:53:39.972396 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 16:53:39.972432 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:53:39.974602 (+ 2170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:53:39.974603 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:53:39.974605 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:53:39.974677 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:53:39.974679 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:53:39.975812 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:53:39.975813 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:53:39.976854 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:53:39.976857 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:53:39.977798 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:53:39.977799 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:53:39.977859 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:53:39.977863 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 16:53:39.977927 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 16:53:39.977965 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":101,"threads_started":1}
I20260808 16:53:39.978051 26922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:53:39.972335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44862 (local address 127.8.24.2:42815)
0808 16:53:39.974468 (+ 2133us) server_negotiation.cc:206] Beginning negotiation
0808 16:53:39.974472 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 16:53:39.974480 (+ 8us) server_negotiation.cc:407] Connection header received
0808 16:53:39.974510 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:53:39.974512 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:53:39.974524 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:53:39.974592 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:53:39.975027 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:53:39.975731 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:53:39.976952 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:53:39.977709 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:53:39.977780 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:53:39.977846 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 16:53:39.977940 (+ 94us) server_negotiation.cc:299] Negotiation successful
0808 16:53:39.977972 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2098,"thread_start_us":79,"threads_started":1}
I20260808 16:53:39.978763 26579 tablet_service.cc:1467] Processing CreateTablet for tablet 05abe5347c614a6388289226ca7d2bc4 (DEFAULT_TABLE table=test-table [id=f4b1e19d808e4ebbaa7923ac3d20d92a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:53:39.979053 26579 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05abe5347c614a6388289226ca7d2bc4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:53:39.980429 26920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:53:39.972293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.3:44031 (local address 127.0.0.1:48108)
0808 16:53:39.974695 (+ 2402us) negotiation.cc:107] Waiting for socket to connect
0808 16:53:39.974704 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 16:53:39.974751 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:53:39.975819 (+ 1068us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:53:39.975820 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:53:39.975822 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:53:39.975865 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:53:39.975866 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:53:39.978159 (+ 2293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:53:39.978160 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:53:39.979281 (+ 1121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:53:39.979285 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:53:39.980200 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:53:39.980201 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:53:39.980269 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:53:39.980273 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 16:53:39.980326 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 16:53:39.980362 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2385,"thread_start_us":67,"threads_started":1}
I20260808 16:53:39.980454 26924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:53:39.972482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48108 (local address 127.8.24.3:44031)
0808 16:53:39.972639 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0808 16:53:39.972643 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 16:53:39.974794 (+ 2151us) server_negotiation.cc:407] Connection header received
0808 16:53:39.974952 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:53:39.974955 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:53:39.974970 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:53:39.975037 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:53:39.976949 (+ 1912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:53:39.977981 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:53:39.979395 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:53:39.980130 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:53:39.980198 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:53:39.980271 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 16:53:39.980338 (+ 67us) server_negotiation.cc:299] Negotiation successful
0808 16:53:39.980375 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":82,"threads_started":1}
I20260808 16:53:39.981766 26929 tablet_bootstrap.cc:492] T 05abe5347c614a6388289226ca7d2bc4 P 25de29a6a6924edfafa087050138b7b4: Bootstrap starting.
I20260808 16:53:39.982374 26787 tablet_service.cc:1467] Processing CreateTablet for tablet 05abe5347c614a6388289226ca7d2bc4 (DEFAULT_TABLE table=test-table [id=f4b1e19d808e4ebbaa7923ac3d20d92a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:53:39.982522 26926 tablet_bootstrap.cc:492] T 05abe5347c614a6388289226ca7d2bc4 P d9f52b7084da4804b09a44ec028fb18d: Bootstrap starting.
I20260808 16:53:39.982681 26787 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05abe5347c614a6388289226ca7d2bc4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:53:39.982954 26929 tablet_bootstrap.cc:654] T 05abe5347c614a6388289226ca7d2bc4 P 25de29a6a6924edfafa087050138b7b4: Neither blocks nor log segments found. Creating new log.
I20260808 16:53:39.983247 26926 tablet_bootstrap.cc:654] T 05abe5347c614a6388289226ca7d2bc4 P d9f52b7084da4804b09a44ec028fb18d: Neither blocks nor log segments found. Creating new log.
I20260808 16:53:39.983543 26926 log.cc:824] T 05abe5347c614a6388289226ca7d2bc4 P d9f52b7084da4804b09a44ec028fb18d: Log is configured to *not* fsync() on all Append() calls
I20260808 16:53:39.983583 26929 log.cc:824] T 05abe5347c614a6388289226ca7d2bc4 P 25de29a6a6924edfafa087050138b7b4: Log is configured to *not* fsync() on all Append() calls
I20260808 16:53:39.985749 26934 tablet_bootstrap.cc:492] T 05abe5347c614a6388289226ca7d2bc4 P 41e9225a2ecb45d0969cd8c5d07bdff9: Bootstrap starting.
I20260808 16:53:39.986790 26934 tablet_bootstrap.cc:654] T 05abe5347c614a6388289226ca7d2bc4 P 41e9225a2ecb45d0969cd8c5d07bdff9: Neither blocks nor log segments found. Creating new log.
I20260808 16:53:39.986959 26926 tablet_bootstrap.cc:492] T 05abe5347c614a6388289226ca7d2bc4 P d9f52b7084da4804b09a44ec028fb18d: No bootstrap required, opened a new log
I20260808 16:53:39.987023 26926 ts_tablet_manager.cc:1397] T 05abe5347c614a6388289226ca7d2bc4 P d9f52b7084da4804b09a44ec028fb18d: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260808 16:53:39.987130 26929 tablet_bootstrap.cc:492] T 05abe5347c614a6388289226ca7d2bc4 P 25de29a6a6924edfafa087050138b7b4: No bootstrap required, opened a new log
I20260808 16:53:39.987142 26934 log.cc:824] T 05abe5347c614a6388289226ca7d2bc4 P 41e9225a2ecb45d0969cd8c5d07bdff9: Log is configured to *not* fsync() on all Append() calls
I20260808 16:53:39.987200 26929 ts_tablet_manager.cc:1397] T 05abe5347c614a6388289226ca7d2bc4 P 25de29a6a6924edfafa087050138b7b4: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260808 16:53:39.987808 26934 tablet_bootstrap.cc:492] T 05abe5347c614a6388289226ca7d2bc4 P 41e9225a2ecb45d0969cd8c5d07bdff9: No bootstrap required, opened a new log
I20260808 16:53:39.987864 26934 ts_tablet_manager.cc:1397] T 05abe5347c614a6388289226ca7d2bc4 P 41e9225a2ecb45d0969cd8c5d07bdff9: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 16:53:39.989096 26929 raft_consensus.cc:348] T 05abe5347c614a6388289226ca7d2bc4 P 25de29a6a6924edfafa087050138b7b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9f52b7084da4804b09a44ec028fb18d" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 43351 } } peers { permanent_uuid: "25de29a6a6924edfafa087050138b7b4" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 42815 } } peers { permanent_uuid: "41e9225a2ecb45d0969cd8c5d07bdff9" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 44031 } }
I20260808 16:53:39.989281 26929 raft_consensus.cc:374] T 05abe5347c614a6388289226ca7d2bc4 P 25de29a6a6924edfafa087050138b7b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:53:39.989313 26929 raft_consensus.cc:729] T 05abe5347c614a6388289226ca7d2bc4 P 25de29a6a6924edfafa087050138b7b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25de29a6a6924edfafa087050138b7b4, State: Initialized, Role: FOLLOWER
I20260808 16:53:39.989609 26926 raft_consensus.cc:348] T 05abe5347c614a6388289226ca7d2bc4 P d9f52b7084da4804b09a44ec028fb18d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9f52b7084da4804b09a44ec028fb18d" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 43351 } } peers { permanent_uuid: "25de29a6a6924edfafa087050138b7b4" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 42815 } } peers { permanent_uuid: "41e9225a2ecb45d0969cd8c5d07bdff9" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 44031 } }
I20260808 16:53:39.989768 26926 raft_consensus.cc:374] T 05abe5347c614a6388289226ca7d2bc4 P d9f52b7084da4804b09a44ec028fb18d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:53:39.989723 26929 consensus_queue.cc:260] T 05abe5347c614a6388289226ca7d2bc4 P 25de29a6a6924edfafa087050138b7b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9f52b7084da4804b09a44ec028fb18d" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 43351 } } peers { permanent_uuid: "25de29a6a6924edfafa087050138b7b4" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 42815 } } peers { permanent_uuid: "41e9225a2ecb45d0969cd8c5d07bdff9" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 44031 } }
I20260808 16:53:39.989800 26926 raft_consensus.cc:729] T 05abe5347c614a6388289226ca7d2bc4 P d9f52b7084da4804b09a44ec028fb18d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9f52b7084da4804b09a44ec028fb18d, State: Initialized, Role: FOLLOWER
I20260808 16:53:39.989702 26934 raft_consensus.cc:348] T 05abe5347c614a6388289226ca7d2bc4 P 41e9225a2ecb45d0969cd8c5d07bdff9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9f52b7084da4804b09a44ec028fb18d" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 43351 } } peers { permanent_uuid: "25de29a6a6924edfafa087050138b7b4" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 42815 } } peers { permanent_uuid: "41e9225a2ecb45d0969cd8c5d07bdff9" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 44031 } }
I20260808 16:53:39.989840 26934 raft_consensus.cc:374] T 05abe5347c614a6388289226ca7d2bc4 P 41e9225a2ecb45d0969cd8c5d07bdff9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:53:39.989876 26934 raft_consensus.cc:729] T 05abe5347c614a6388289226ca7d2bc4 P 41e9225a2ecb45d0969cd8c5d07bdff9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41e9225a2ecb45d0969cd8c5d07bdff9, State: Initialized, Role: FOLLOWER
I20260808 16:53:39.989914 26926 consensus_queue.cc:260] T 05abe5347c614a6388289226ca7d2bc4 P d9f52b7084da4804b09a44ec028fb18d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9f52b7084da4804b09a44ec028fb18d" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 43351 } } peers { permanent_uuid: "25de29a6a6924edfafa087050138b7b4" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 42815 } } peers { permanent_uuid: "41e9225a2ecb45d0969cd8c5d07bdff9" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 44031 } }
I20260808 16:53:39.989997 26929 ts_tablet_manager.cc:1428] T 05abe5347c614a6388289226ca7d2bc4 P 25de29a6a6924edfafa087050138b7b4: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 16:53:39.989970 26934 consensus_queue.cc:260] T 05abe5347c614a6388289226ca7d2bc4 P 41e9225a2ecb45d0969cd8c5d07bdff9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9f52b7084da4804b09a44ec028fb18d" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 43351 } } peers { permanent_uuid: "25de29a6a6924edfafa087050138b7b4" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 42815 } } peers { permanent_uuid: "41e9225a2ecb45d0969cd8c5d07bdff9" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 44031 } }
I20260808 16:53:39.990103 26680 heartbeater.cc:503] Master 127.8.24.62:40269 was elected leader, sending a full tablet report...
I20260808 16:53:39.990187 26934 ts_tablet_manager.cc:1428] T 05abe5347c614a6388289226ca7d2bc4 P 41e9225a2ecb45d0969cd8c5d07bdff9: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 16:53:39.990190 26926 ts_tablet_manager.cc:1428] T 05abe5347c614a6388289226ca7d2bc4 P d9f52b7084da4804b09a44ec028fb18d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 16:53:39.990296 26475 heartbeater.cc:503] Master 127.8.24.62:40269 was elected leader, sending a full tablet report...
I20260808 16:53:39.990309 26899 heartbeater.cc:503] Master 127.8.24.62:40269 was elected leader, sending a full tablet report...
Aug 08 16:53:40 dist-test-slave-6zbq krb5kdc[25375](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208019, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for HTTP/127.8.24.20@KRBTEST.COM
W20260808 16:53:40.171057 26476 tablet.cc:2369] T 05abe5347c614a6388289226ca7d2bc4 P d9f52b7084da4804b09a44ec028fb18d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 16:53:40.183647 26901 tablet.cc:2369] T 05abe5347c614a6388289226ca7d2bc4 P 41e9225a2ecb45d0969cd8c5d07bdff9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 16:53:40.185624 26681 tablet.cc:2369] T 05abe5347c614a6388289226ca7d2bc4 P 25de29a6a6924edfafa087050138b7b4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 16:53:40.200096 26940 raft_consensus.cc:482] T 05abe5347c614a6388289226ca7d2bc4 P 41e9225a2ecb45d0969cd8c5d07bdff9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 16:53:40.200215 26940 raft_consensus.cc:504] T 05abe5347c614a6388289226ca7d2bc4 P 41e9225a2ecb45d0969cd8c5d07bdff9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9f52b7084da4804b09a44ec028fb18d" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 43351 } } peers { permanent_uuid: "25de29a6a6924edfafa087050138b7b4" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 42815 } } peers { permanent_uuid: "41e9225a2ecb45d0969cd8c5d07bdff9" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 44031 } }
I20260808 16:53:40.200590 26940 leader_election.cc:288] T 05abe5347c614a6388289226ca7d2bc4 P 41e9225a2ecb45d0969cd8c5d07bdff9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d9f52b7084da4804b09a44ec028fb18d (127.8.24.1:43351), 25de29a6a6924edfafa087050138b7b4 (127.8.24.2:42815)
I20260808 16:53:40.204542 26902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:53:40.200746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:43351 (local address 127.8.24.3:36205)
0808 16:53:40.200849 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0808 16:53:40.200861 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 16:53:40.200904 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:53:40.201203 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:53:40.201204 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:53:40.201206 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:53:40.201279 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:53:40.201281 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:53:40.202170 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:53:40.202172 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:53:40.203420 (+ 1248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:53:40.203424 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:53:40.204280 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:53:40.204281 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:53:40.204361 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:53:40.204365 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 16:53:40.204413 (+ 48us) client_negotiation.cc:240] Negotiation successful
0808 16:53:40.204456 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":81}
I20260808 16:53:40.204641 26922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:53:40.201061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:51083 (local address 127.8.24.2:42815)
0808 16:53:40.201120 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 16:53:40.201122 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:53:40.201174 (+ 52us) server_negotiation.cc:407] Connection header received
0808 16:53:40.201270 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:53:40.201271 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:53:40.201282 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:53:40.201308 (+ 26us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:53:40.201541 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:53:40.202295 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:53:40.203757 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:53:40.204294 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:53:40.204334 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:53:40.204343 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 16:53:40.204533 (+ 190us) server_negotiation.cc:299] Negotiation successful
0808 16:53:40.204565 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 16:53:40.204841 26941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:53:40.200850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.2:42815 (local address 127.8.24.3:51083)
0808 16:53:40.201111 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0808 16:53:40.201120 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 16:53:40.201201 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:53:40.201368 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:53:40.201368 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:53:40.201370 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:53:40.201444 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:53:40.201445 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:53:40.202455 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:53:40.202456 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:53:40.203661 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:53:40.203664 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:53:40.204395 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:53:40.204395 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:53:40.204465 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:53:40.204468 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 16:53:40.204715 (+ 247us) client_negotiation.cc:240] Negotiation successful
0808 16:53:40.204757 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":103,"threads_started":1}
I20260808 16:53:40.205132 26925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:53:40.200979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:36205 (local address 127.8.24.1:43351)
0808 16:53:40.201023 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 16:53:40.201025 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:53:40.201034 (+ 9us) server_negotiation.cc:407] Connection header received
0808 16:53:40.201060 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:53:40.201061 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:53:40.201072 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:53:40.201139 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:53:40.201362 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:53:40.202055 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:53:40.203536 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:53:40.204162 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:53:40.204233 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:53:40.204246 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0808 16:53:40.205019 (+ 773us) server_negotiation.cc:299] Negotiation successful
0808 16:53:40.205069 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 16:53:40.205199 26605 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05abe5347c614a6388289226ca7d2bc4" candidate_uuid: "41e9225a2ecb45d0969cd8c5d07bdff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25de29a6a6924edfafa087050138b7b4" is_pre_election: true
I20260808 16:53:40.205333 26605 raft_consensus.cc:2454] T 05abe5347c614a6388289226ca7d2bc4 P 25de29a6a6924edfafa087050138b7b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41e9225a2ecb45d0969cd8c5d07bdff9 in term 0.
I20260808 16:53:40.205384 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05abe5347c614a6388289226ca7d2bc4" candidate_uuid: "41e9225a2ecb45d0969cd8c5d07bdff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9f52b7084da4804b09a44ec028fb18d" is_pre_election: true
I20260808 16:53:40.205526 26401 raft_consensus.cc:2454] T 05abe5347c614a6388289226ca7d2bc4 P d9f52b7084da4804b09a44ec028fb18d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41e9225a2ecb45d0969cd8c5d07bdff9 in term 0.
I20260808 16:53:40.205554 26715 leader_election.cc:302] T 05abe5347c614a6388289226ca7d2bc4 P 41e9225a2ecb45d0969cd8c5d07bdff9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25de29a6a6924edfafa087050138b7b4, 41e9225a2ecb45d0969cd8c5d07bdff9; no voters:
I20260808 16:53:40.205682 26940 raft_consensus.cc:2781] T 05abe5347c614a6388289226ca7d2bc4 P 41e9225a2ecb45d0969cd8c5d07bdff9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 16:53:40.205725 26940 raft_consensus.cc:482] T 05abe5347c614a6388289226ca7d2bc4 P 41e9225a2ecb45d0969cd8c5d07bdff9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 16:53:40.205744 26940 raft_consensus.cc:3037] T 05abe5347c614a6388289226ca7d2bc4 P 41e9225a2ecb45d0969cd8c5d07bdff9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:53:40.206597 26940 raft_consensus.cc:504] T 05abe5347c614a6388289226ca7d2bc4 P 41e9225a2ecb45d0969cd8c5d07bdff9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9f52b7084da4804b09a44ec028fb18d" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 43351 } } peers { permanent_uuid: "25de29a6a6924edfafa087050138b7b4" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 42815 } } peers { permanent_uuid: "41e9225a2ecb45d0969cd8c5d07bdff9" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 44031 } }
I20260808 16:53:40.206732 26940 leader_election.cc:288] T 05abe5347c614a6388289226ca7d2bc4 P 41e9225a2ecb45d0969cd8c5d07bdff9 [CANDIDATE]: Term 1 election: Requested vote from peers d9f52b7084da4804b09a44ec028fb18d (127.8.24.1:43351), 25de29a6a6924edfafa087050138b7b4 (127.8.24.2:42815)
I20260808 16:53:40.206894 26605 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05abe5347c614a6388289226ca7d2bc4" candidate_uuid: "41e9225a2ecb45d0969cd8c5d07bdff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25de29a6a6924edfafa087050138b7b4"
I20260808 16:53:40.206913 26401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05abe5347c614a6388289226ca7d2bc4" candidate_uuid: "41e9225a2ecb45d0969cd8c5d07bdff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9f52b7084da4804b09a44ec028fb18d"
I20260808 16:53:40.206964 26605 raft_consensus.cc:3037] T 05abe5347c614a6388289226ca7d2bc4 P 25de29a6a6924edfafa087050138b7b4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:53:40.206985 26401 raft_consensus.cc:3037] T 05abe5347c614a6388289226ca7d2bc4 P d9f52b7084da4804b09a44ec028fb18d [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:53:40.207749 26605 raft_consensus.cc:2454] T 05abe5347c614a6388289226ca7d2bc4 P 25de29a6a6924edfafa087050138b7b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41e9225a2ecb45d0969cd8c5d07bdff9 in term 1.
I20260808 16:53:40.207872 26401 raft_consensus.cc:2454] T 05abe5347c614a6388289226ca7d2bc4 P d9f52b7084da4804b09a44ec028fb18d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41e9225a2ecb45d0969cd8c5d07bdff9 in term 1.
I20260808 16:53:40.207934 26715 leader_election.cc:302] T 05abe5347c614a6388289226ca7d2bc4 P 41e9225a2ecb45d0969cd8c5d07bdff9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25de29a6a6924edfafa087050138b7b4, 41e9225a2ecb45d0969cd8c5d07bdff9; no voters:
I20260808 16:53:40.208003 26940 raft_consensus.cc:2781] T 05abe5347c614a6388289226ca7d2bc4 P 41e9225a2ecb45d0969cd8c5d07bdff9 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:53:40.208139 26940 raft_consensus.cc:686] T 05abe5347c614a6388289226ca7d2bc4 P 41e9225a2ecb45d0969cd8c5d07bdff9 [term 1 LEADER]: Becoming Leader. State: Replica: 41e9225a2ecb45d0969cd8c5d07bdff9, State: Running, Role: LEADER
I20260808 16:53:40.208225 26940 consensus_queue.cc:237] T 05abe5347c614a6388289226ca7d2bc4 P 41e9225a2ecb45d0969cd8c5d07bdff9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9f52b7084da4804b09a44ec028fb18d" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 43351 } } peers { permanent_uuid: "25de29a6a6924edfafa087050138b7b4" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 42815 } } peers { permanent_uuid: "41e9225a2ecb45d0969cd8c5d07bdff9" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 44031 } }
I20260808 16:53:40.209028 26118 catalog_manager.cc:5495] T 05abe5347c614a6388289226ca7d2bc4 P 41e9225a2ecb45d0969cd8c5d07bdff9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 41e9225a2ecb45d0969cd8c5d07bdff9 (127.8.24.3). New cstate: current_term: 1 leader_uuid: "41e9225a2ecb45d0969cd8c5d07bdff9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9f52b7084da4804b09a44ec028fb18d" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 43351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25de29a6a6924edfafa087050138b7b4" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 42815 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41e9225a2ecb45d0969cd8c5d07bdff9" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 44031 } health_report { overall_health: HEALTHY } } }
I20260808 16:53:40.258046 26924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:53:40.255230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48118 (local address 127.8.24.3:44031)
0808 16:53:40.255282 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 16:53:40.255284 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:53:40.255297 (+ 13us) server_negotiation.cc:407] Connection header received
0808 16:53:40.255334 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:53:40.255335 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:53:40.255357 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:53:40.255444 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 16:53:40.255550 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:53:40.256225 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:53:40.257345 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:53:40.257546 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:53:40.257602 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:53:40.257750 (+ 148us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 16:53:40.257848 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 16:53:40.257918 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 16:53:40.257956 (+ 38us) server_negotiation.cc:299] Negotiation successful
0808 16:53:40.257975 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 16:53:40.259869 26605 raft_consensus.cc:1261] T 05abe5347c614a6388289226ca7d2bc4 P 25de29a6a6924edfafa087050138b7b4 [term 1 FOLLOWER]: Refusing update from remote peer 41e9225a2ecb45d0969cd8c5d07bdff9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:53:40.259898 26401 raft_consensus.cc:1261] T 05abe5347c614a6388289226ca7d2bc4 P d9f52b7084da4804b09a44ec028fb18d [term 1 FOLLOWER]: Refusing update from remote peer 41e9225a2ecb45d0969cd8c5d07bdff9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:53:40.260229 26940 consensus_queue.cc:1035] T 05abe5347c614a6388289226ca7d2bc4 P 41e9225a2ecb45d0969cd8c5d07bdff9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9f52b7084da4804b09a44ec028fb18d" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 43351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:53:40.260413 26945 consensus_queue.cc:1035] T 05abe5347c614a6388289226ca7d2bc4 P 41e9225a2ecb45d0969cd8c5d07bdff9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25de29a6a6924edfafa087050138b7b4" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 42815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:53:40.264889 26951 mvcc.cc:205] Tried to move back new op lower bound from 7316308050981990400 to 7316308050773344256. Current Snapshot: MvccSnapshot[applied={T|T < 7316308050981990400}]
I20260808 16:53:40.264986 26948 mvcc.cc:205] Tried to move back new op lower bound from 7316308050981990400 to 7316308050773344256. Current Snapshot: MvccSnapshot[applied={T|T < 7316308050981990400 or (T in {7316308050981990400})}]
I20260808 16:53:40.265728 26117 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48308:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 16:53:40.265825 26117 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48308:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 16:53:40.265861 26950 mvcc.cc:205] Tried to move back new op lower bound from 7316308050981990400 to 7316308050773344256. Current Snapshot: MvccSnapshot[applied={T|T < 7316308050981990400}]
I20260808 16:53:40.266669 26117 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 30126cedc2a64c66b655b04a7d623041: Sending DeleteTablet for 3 replicas of tablet 05abe5347c614a6388289226ca7d2bc4
I20260808 16:53:40.267015 26369 tablet_service.cc:1514] Processing DeleteTablet for tablet 05abe5347c614a6388289226ca7d2bc4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:53:40 UTC) from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:59434
I20260808 16:53:40.267056 26787 tablet_service.cc:1514] Processing DeleteTablet for tablet 05abe5347c614a6388289226ca7d2bc4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:53:40 UTC) from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:48108
I20260808 16:53:40.267045 26579 tablet_service.cc:1514] Processing DeleteTablet for tablet 05abe5347c614a6388289226ca7d2bc4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:53:40 UTC) from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:44862
I20260808 16:53:40.267414 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 26257
I20260808 16:53:40.274435 26967 tablet_replica.cc:330] T 05abe5347c614a6388289226ca7d2bc4 P 25de29a6a6924edfafa087050138b7b4: stopping tablet replica
I20260808 16:53:40.274530 26967 raft_consensus.cc:2229] T 05abe5347c614a6388289226ca7d2bc4 P 25de29a6a6924edfafa087050138b7b4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 16:53:40.274612 26967 raft_consensus.cc:2258] T 05abe5347c614a6388289226ca7d2bc4 P 25de29a6a6924edfafa087050138b7b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:53:40.277439 26967 ts_tablet_manager.cc:1905] T 05abe5347c614a6388289226ca7d2bc4 P 25de29a6a6924edfafa087050138b7b4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:53:40.279471 26967 ts_tablet_manager.cc:1918] T 05abe5347c614a6388289226ca7d2bc4 P 25de29a6a6924edfafa087050138b7b4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 16:53:40.279551 26967 log.cc:1196] T 05abe5347c614a6388289226ca7d2bc4 P 25de29a6a6924edfafa087050138b7b4: Deleting WAL directory at /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/minicluster-data/ts-1/wal/wals/05abe5347c614a6388289226ca7d2bc4
I20260808 16:53:40.279865 26967 ts_tablet_manager.cc:1939] T 05abe5347c614a6388289226ca7d2bc4 P 25de29a6a6924edfafa087050138b7b4: Deleting consensus metadata
I20260808 16:53:40.280408 26966 tablet_replica.cc:330] T 05abe5347c614a6388289226ca7d2bc4 P 41e9225a2ecb45d0969cd8c5d07bdff9: stopping tablet replica
I20260808 16:53:40.280524 26966 raft_consensus.cc:2229] T 05abe5347c614a6388289226ca7d2bc4 P 41e9225a2ecb45d0969cd8c5d07bdff9 [term 1 LEADER]: Raft consensus shutting down.
I20260808 16:53:40.280661 26966 raft_consensus.cc:2258] T 05abe5347c614a6388289226ca7d2bc4 P 41e9225a2ecb45d0969cd8c5d07bdff9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:53:40.283949 26096 catalog_manager.cc:4841] TS 25de29a6a6924edfafa087050138b7b4 (127.8.24.2:42815): tablet 05abe5347c614a6388289226ca7d2bc4 (table test-table [id=f4b1e19d808e4ebbaa7923ac3d20d92a]) successfully deleted
I20260808 16:53:40.290719 26966 ts_tablet_manager.cc:1905] T 05abe5347c614a6388289226ca7d2bc4 P 41e9225a2ecb45d0969cd8c5d07bdff9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:53:40.292663 26966 ts_tablet_manager.cc:1918] T 05abe5347c614a6388289226ca7d2bc4 P 41e9225a2ecb45d0969cd8c5d07bdff9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 16:53:40.292753 26966 log.cc:1196] T 05abe5347c614a6388289226ca7d2bc4 P 41e9225a2ecb45d0969cd8c5d07bdff9: Deleting WAL directory at /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786207919384338-8288-0/minicluster-data/ts-2/wal/wals/05abe5347c614a6388289226ca7d2bc4
I20260808 16:53:40.293089 26966 ts_tablet_manager.cc:1939] T 05abe5347c614a6388289226ca7d2bc4 P 41e9225a2ecb45d0969cd8c5d07bdff9: Deleting consensus metadata
I20260808 16:53:40.293577 26096 catalog_manager.cc:4841] TS 41e9225a2ecb45d0969cd8c5d07bdff9 (127.8.24.3:44031): tablet 05abe5347c614a6388289226ca7d2bc4 (table test-table [id=f4b1e19d808e4ebbaa7923ac3d20d92a]) successfully deleted
W20260808 16:53:40.305174 26091 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.24.1:43351 (error 108)
I20260808 16:53:40.305822 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 26484
I20260808 16:53:40.313880 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 26687
I20260808 16:53:40.329052 26920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:53:40.328895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:43351 (local address 127.0.0.1:59438)
0808 16:53:40.328959 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0808 16:53:40.329012 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.1:43351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260808 16:53:40.329308 26091 catalog_manager.cc:4584] TS d9f52b7084da4804b09a44ec028fb18d (127.8.24.1:43351): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 05abe5347c614a6388289226ca7d2bc4: Network error: Client connection negotiation failed: client connection to 127.8.24.1:43351: connect: Connection refused (error 111)
I20260808 16:53:40.334180 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 26066
I20260808 16:53:40.342417 8288 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260808 16:53:40.675693 8288 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260808 16:53:40.675787 8288 mini_ranger.cc:66] Stopping Ranger...
I20260808 16:53:41.021191 8288 mini_ranger.cc:68] Stopped Ranger
2026-08-08 16:53:41.021 UTC [25420] LOG: received smart shutdown request
2026-08-08 16:53:41.024 UTC [25420] LOG: background worker "logical replication launcher" (PID 25434) exited with exit code 1
2026-08-08 16:53:41.024 UTC [25426] LOG: shutting down
2026-08-08 16:53:41.144 UTC [25420] LOG: database system is shut down
2026-08-08T16:53:41Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (43087 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 16:53:41 dist-test-slave-6zbq krb5kdc[26980](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 16:53:41 dist-test-slave-6zbq krb5kdc[26980](info): set up 2 sockets
Aug 08 16:53:41 dist-test-slave-6zbq krb5kdc[26980](info): commencing operation
krb5kdc: starting...
W20260808 16:53:42.358812 8288 mini_kdc.cc:121] Time spent starting KDC: real 1.108s user 0.001s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/krb5kdc/test-user.keytab.
Aug 08 16:53:42 dist-test-slave-6zbq krb5kdc[26980](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208022, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T16:53:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T16:53:42Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.8.24.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.8.24.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.8.24.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/krb5kdc/rangeradmin_127.8.24.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.8.24.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/krb5kdc/rangeradmin_127.8.24.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.8.24.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.8.24.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.8.24.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/krb5kdc/rangerlookup_127.8.24.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.8.24.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/krb5kdc/rangerlookup_127.8.24.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.8.24.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.8.24.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.8.24.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/krb5kdc/HTTP_127.8.24.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.8.24.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/krb5kdc/HTTP_127.8.24.20@KRBTEST.COM.keytab.
I20260808 16:53:42.506139 8288 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskJykUMM/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/postgres -l logfile start
2026-08-08 16:53:44.604 UTC [27027] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-08 16:53:44.604 UTC [27027] LOG: listening on IPv4 address "127.8.24.20", port 53855
2026-08-08 16:53:44.607 UTC [27027] LOG: listening on Unix socket "/tmp/.s.PGSQL.53855"
2026-08-08 16:53:44.626 UTC [27030] LOG: database system was shut down at 2026-08-08 16:53:43 UTC
2026-08-08 16:53:44.629 UTC [27027] LOG: database system is ready to accept connections
I20260808 16:53:45.674571 8288 mini_postgres.cc:96] Postgres bound to 53855
2026-08-08 16:53:45.681 UTC [27042] FATAL: database "slave" does not exist
127.8.24.20:53855 - accepting connections
I20260808 16:53:45.681501 8288 mini_ranger.cc:161] Starting Ranger...
I20260808 16:53:45.694998 8288 mini_ranger.cc:84] Created miniranger Postgres user
I20260808 16:53:46.039819 8288 mini_ranger.cc:87] Created ranger Postgres database
I20260808 16:53:46.039885 8288 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ranger-admin
2026-08-08 16:53:46,298 [I] DB FLAVOR :POSTGRES
2026-08-08 16:53:46,299 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 16:53:46,299 [I] Checking connection..
2026-08-08 16:53:46,299 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJykUMM/build/release/bin/postgresql.jar:/tmp/dist-test-taskJykUMM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.24.20:53855/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-08 16:53:46,549 [I] Checking connection passed.
2026-08-08 16:53:46,549 [I] --------- Verifying version history table ---------
2026-08-08 16:53:46,549 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJykUMM/build/release/bin/postgresql.jar:/tmp/dist-test-taskJykUMM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.24.20:53855/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 16:53:46,796 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 16:53:46,797 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJykUMM/build/release/bin/postgresql.jar:/tmp/dist-test-taskJykUMM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.24.20:53855/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 16:53:47,045 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 16:53:47,045 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-08 16:53:47,046 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJykUMM/build/release/bin/postgresql.jar:/tmp/dist-test-taskJykUMM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.24.20:53855/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskJykUMM/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-08 16:53:47.262 UTC [27134] WARNING: there is no transaction in progress
2026-08-08 16:53:47,268 [I] create_dbversion_catalog.sql file imported successfully
2026-08-08 16:53:47,268 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJykUMM/build/release/bin/postgresql.jar:/tmp/dist-test-taskJykUMM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.24.20:53855/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 16:53:47,517 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-08 16:53:47,517 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 16:53:47,517 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJykUMM/build/release/bin/postgresql.jar:/tmp/dist-test-taskJykUMM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.24.20:53855/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-08 16:53:47,751 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJykUMM/build/release/bin/postgresql.jar:/tmp/dist-test-taskJykUMM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.24.20:53855/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-08 16:53:47,997 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJykUMM/build/release/bin/postgresql.jar:/tmp/dist-test-taskJykUMM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.24.20:53855/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-6zbq.c.gcp-upstream.internal','N') ;"
2026-08-08 16:53:48,237 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJykUMM/build/release/bin/postgresql.jar:/tmp/dist-test-taskJykUMM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.24.20:53855/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 16:53:48,481 [I] Table x_portal_user does not exist in database ranger
2026-08-08 16:53:48,481 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJykUMM/build/release/bin/postgresql.jar:/tmp/dist-test-taskJykUMM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.24.20:53855/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 16:53:48,720 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-08 16:53:48,720 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJykUMM/build/release/bin/postgresql.jar:/tmp/dist-test-taskJykUMM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.24.20:53855/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 16:53:48,944 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJykUMM/build/release/bin/postgresql.jar:/tmp/dist-test-taskJykUMM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.24.20:53855/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 16:53:49,189 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-08 16:53:49,190 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJykUMM/build/release/bin/postgresql.jar:/tmp/dist-test-taskJykUMM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.24.20:53855/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskJykUMM/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-08 16:53:50.005 UTC [27311] WARNING: there is no transaction in progress
2026-08-08 16:53:50.019 UTC [27311] WARNING: there is no transaction in progress
2026-08-08 16:53:50.033 UTC [27311] WARNING: there is no transaction in progress
2026-08-08 16:53:50.046 UTC [27311] WARNING: there is no transaction in progress
2026-08-08 16:53:50.060 UTC [27311] WARNING: there is no transaction in progress
2026-08-08 16:53:50.075 UTC [27311] WARNING: there is no transaction in progress
2026-08-08 16:53:50.124 UTC [27311] WARNING: there is no transaction in progress
2026-08-08 16:53:50.129 UTC [27311] WARNING: there is no transaction in progress
2026-08-08 16:53:50.141 UTC [27311] WARNING: there is no transaction in progress
2026-08-08 16:53:50.150 UTC [27311] WARNING: there is no transaction in progress
2026-08-08 16:53:50.159 UTC [27311] WARNING: there is no transaction in progress
2026-08-08 16:53:50.170 UTC [27311] WARNING: there is no transaction in progress
2026-08-08 16:53:50.178 UTC [27311] WARNING: there is no transaction in progress
2026-08-08 16:53:50.183 UTC [27311] WARNING: there is no transaction in progress
2026-08-08 16:53:50.189 UTC [27311] WARNING: there is no transaction in progress
2026-08-08 16:53:50.194 UTC [27311] WARNING: there is no transaction in progress
2026-08-08 16:53:50,682 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-08 16:53:50,683 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJykUMM/build/release/bin/postgresql.jar:/tmp/dist-test-taskJykUMM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.24.20:53855/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-08 16:53:50,919 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-08 16:53:50,920 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJykUMM/build/release/bin/postgresql.jar:/tmp/dist-test-taskJykUMM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.24.20:53855/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 16:53:51,159 [I] Table x_portal_user already exists in database 'ranger'
2026-08-08 16:53:51,160 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJykUMM/build/release/bin/postgresql.jar:/tmp/dist-test-taskJykUMM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.24.20:53855/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 16:53:51,400 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-08 16:53:51,400 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJykUMM/build/release/bin/postgresql.jar:/tmp/dist-test-taskJykUMM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.24.20:53855/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 16:53:51,637 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJykUMM/build/release/bin/postgresql.jar:/tmp/dist-test-taskJykUMM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.24.20:53855/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 16:53:51,883 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJykUMM/build/release/bin/postgresql.jar:/tmp/dist-test-taskJykUMM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.24.20:53855/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-6zbq.c.gcp-upstream.internal';"
2026-08-08 16:53:52,097 [I] CORE_DB_SCHEMA import status has been updated
2026-08-08 16:53:52,097 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJykUMM/build/release/bin/postgresql.jar:/tmp/dist-test-taskJykUMM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.24.20:53855/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-08 16:53:52,336 [I] DB_PATCHES have already been applied
I20260808 16:53:52.341514 8288 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ranger-admin:/tmp/dist-test-taskJykUMM/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskJykUMM/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskJykUMM/build/release/bin/postgresql.jar:/tmp/dist-test-taskJykUMM/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260808 16:53:52.341571 8288 mini_ranger.cc:193] Using host: 127.8.24.20
I20260808 16:53:52.345175 8288 mini_ranger.cc:227] Ranger admin URL: http://127.8.24.20:41057
Aug 08, 2026 4:53:52 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 4:53:52 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 4:53:52 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ranger-admin/ews/webapp
Aug 08, 2026 4:53:52 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 08, 2026 4:53:52 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 08, 2026 4:53:53 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ranger-admin/ews/webapp].
Aug 08, 2026 4:53:53 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260808 16:53:53.533593 8288 mini_ranger.cc:160] Time spent starting Ranger: real 7.852s user 0.000s sys 0.009s
Aug 08, 2026 4:54:12 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 08, 2026 4:54:12 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 08, 2026 4:54:12 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 08, 2026 4:54:12 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 08, 2026 4:54:12 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 08, 2026 4:54:12 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 08, 2026 4:54:12 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 08, 2026 4:54:12 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 08, 2026 4:54:12 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 08, 2026 4:54:12 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 08, 2026 4:54:12 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 08, 2026 4:54:12 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 08, 2026 4:54:12 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 08, 2026 4:54:12 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 08, 2026 4:54:12 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 08, 2026 4:54:12 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 08, 2026 4:54:12 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 08, 2026 4:54:13 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260808 16:54:14.313961 8288 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.8.24.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.8.24.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.8.24.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/krb5kdc/rangerkms_127.8.24.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.8.24.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/krb5kdc/rangerkms_127.8.24.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.8.24.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.8.24.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.8.24.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/krb5kdc/HTTP_127.8.24.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.8.24.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/krb5kdc/HTTP_127.8.24.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 08 16:54:14 dist-test-slave-6zbq krb5kdc[26980](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208054, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260808 16:54:14.403839 8288 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260808 16:54:14.412887 8288 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260808 16:54:15.061853 8288 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260808 16:54:15.061929 8288 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ranger-kms
I20260808 16:54:15.061944 8288 mini_ranger_kms.cc:227] Using postgres at 127.8.24.20:53855
2026-08-08 16:54:15,208 [I] DB FLAVOR :POSTGRES
2026-08-08 16:54:15,208 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 16:54:15,209 [I] Checking connection
2026-08-08 16:54:15,209 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJykUMM/build/release/bin/postgresql.jar:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.24.20:53855/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-08 16:54:15,433 [I] connection success
2026-08-08 16:54:15,433 [I] --------- Verifying Ranger DB tables ---------
2026-08-08 16:54:15,433 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-08 16:54:15,433 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJykUMM/build/release/bin/postgresql.jar:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.24.20:53855/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-08 16:54:15,652 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-08 16:54:15,652 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 16:54:15,652 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-08 16:54:15,652 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJykUMM/build/release/bin/postgresql.jar:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.24.20:53855/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-08 16:54:15,888 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260808 16:54:16.288414 8288 mini_ranger_kms.cc:310] Created kms service
I20260808 16:54:16.469767 8288 mini_ranger_kms.cc:326] Created kudu user
I20260808 16:54:16.510215 8288 mini_ranger_kms.cc:343] Created rangerkms user
Aug 08 16:54:16 dist-test-slave-6zbq krb5kdc[26980](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208054, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.8.24.20@KRBTEST.COM
I20260808 16:54:16.856618 8288 mini_ranger_kms.cc:384] Added ranger policy
I20260808 16:54:16.856740 8288 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ranger-kms:/tmp/dist-test-taskJykUMM/build/release/bin/postgresql.jar:/tmp/dist-test-taskJykUMM/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskJykUMM/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260808 16:54:16.856757 8288 mini_ranger_kms.cc:240] Using host: 127.8.24.20
I20260808 16:54:16.860668 8288 mini_ranger_kms.cc:276] Ranger KMS PID: 27596
I20260808 16:54:16.860755 8288 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.8.24.20:48301
Aug 08, 2026 4:54:17 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 4:54:17 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 4:54:17 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 08, 2026 4:54:17 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 08, 2026 4:54:18 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ranger-kms/ews/webapp].
Aug 08, 2026 4:54:18 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 08 16:54:18 dist-test-slave-6zbq krb5kdc[26980](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786208058, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.24.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:54:19.313203 8288 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 4.909s user 0.007s sys 0.124s
I20260808 16:54:19.313287 8288 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260808 16:54:19.313334 8288 mini_ranger_kms.cc:401] 127.8.24.20:48301/kms/v1/keys
Aug 08 16:54:19 dist-test-slave-6zbq krb5kdc[26980](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208058, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.24.20@KRBTEST.COM for HTTP/127.8.24.20@KRBTEST.COM
Aug 08 16:54:19 dist-test-slave-6zbq krb5kdc[26980](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208058, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.24.20@KRBTEST.COM for HTTP/127.8.24.20@KRBTEST.COM
Aug 08 16:54:21 dist-test-slave-6zbq krb5kdc[26980](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208061, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.8.24.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
I20260808 16:54:21.977260 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:45771
--webserver_interface=127.8.24.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:42363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:45771
--ranger_config_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.24.20:48301/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:54:22.090581 27682 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:54:22.090831 27682 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:54:22.090862 27682 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:54:22.092440 27682 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 16:54:22.092495 27682 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:54:22.092514 27682 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:54:22.092540 27682 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 16:54:22.092624 27682 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 16:54:22.094873 27682 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:42363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/master-0/wal
--ranger_kms_url=127.8.24.20:48301/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:45771
--ranger_config_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:45771
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.24.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27682
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:54:22.095098 27682 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:54:22.095332 27682 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:54:22.100188 27692 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:54:22.100265 27694 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:54:22.100242 27682 server_base.cc:1034] running on GCE node
W20260808 16:54:22.100200 27690 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:54:22.100633 27682 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:54:22.100939 27682 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:54:22.102103 27682 hybrid_clock.cc:648] HybridClock initialized: now 1786208062102080 us; error 36 us; skew 500 ppm
Aug 08 16:54:22 dist-test-slave-6zbq krb5kdc[26980](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208062, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:54:22.109908 27682 init.cc:377] Logged in from keytab as kudu/127.8.24.62@KRBTEST.COM (short username kudu)
I20260808 16:54:22.110435 27682 webserver.cc:460] Webserver started at http://127.8.24.62:44367/ using document root <none> and password file <none>
I20260808 16:54:22.110675 27682 fs_manager.cc:359] Metadata directory not provided
I20260808 16:54:22.110724 27682 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:54:22.110852 27682 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 16:54:22 dist-test-slave-6zbq krb5kdc[26980](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208062, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for HTTP/127.8.24.20@KRBTEST.COM
I20260808 16:54:22.163051 27682 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/master-0/data/instance:
uuid: "f05881afb75648c89dedd2f5d476e96a"
format_stamp: "Formatted at 2026-08-08 16:54:22 on dist-test-slave-6zbq"
server_key: "2ab9da9c65ba76a224d733803a51cc88"
server_key_iv: "f3b19bc49d74a9075558a7ae2d1dfac6"
server_key_version: "kuduclusterkey@0"
I20260808 16:54:22.163621 27682 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/master-0/wal/instance:
uuid: "f05881afb75648c89dedd2f5d476e96a"
format_stamp: "Formatted at 2026-08-08 16:54:22 on dist-test-slave-6zbq"
server_key: "2ab9da9c65ba76a224d733803a51cc88"
server_key_iv: "f3b19bc49d74a9075558a7ae2d1dfac6"
server_key_version: "kuduclusterkey@0"
I20260808 16:54:22.165638 27682 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.001s
I20260808 16:54:22.178952 27704 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:54:22.179162 27682 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 16:54:22.179265 27682 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/master-0/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/master-0/wal
uuid: "f05881afb75648c89dedd2f5d476e96a"
format_stamp: "Formatted at 2026-08-08 16:54:22 on dist-test-slave-6zbq"
server_key: "2ab9da9c65ba76a224d733803a51cc88"
server_key_iv: "f3b19bc49d74a9075558a7ae2d1dfac6"
server_key_version: "kuduclusterkey@0"
I20260808 16:54:22.179355 27682 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:54:22.213536 27682 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:54:22.215310 27682 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:54:22.215431 27682 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:54:22.219496 27682 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.62:45771
I20260808 16:54:22.219563 27797 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.62:45771 every 8 connection(s)
I20260808 16:54:22.219893 27682 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
I20260808 16:54:22.220536 27800 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:54:22.222815 27800 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f05881afb75648c89dedd2f5d476e96a: Bootstrap starting.
I20260808 16:54:22.223047 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 27682
I20260808 16:54:22.223122 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/master-0/wal/instance
I20260808 16:54:22.223302 8288 external_mini_cluster.cc:1428] Setting key 0093f0b64f905c880efd19aa107be6a2
I20260808 16:54:22.224315 27800 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f05881afb75648c89dedd2f5d476e96a: Neither blocks nor log segments found. Creating new log.
I20260808 16:54:22.224591 27800 log.cc:824] T 00000000000000000000000000000000 P f05881afb75648c89dedd2f5d476e96a: Log is configured to *not* fsync() on all Append() calls
I20260808 16:54:22.225340 27800 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f05881afb75648c89dedd2f5d476e96a: No bootstrap required, opened a new log
I20260808 16:54:22.227878 27800 raft_consensus.cc:348] T 00000000000000000000000000000000 P f05881afb75648c89dedd2f5d476e96a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f05881afb75648c89dedd2f5d476e96a" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 45771 } }
I20260808 16:54:22.228046 27800 raft_consensus.cc:374] T 00000000000000000000000000000000 P f05881afb75648c89dedd2f5d476e96a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:54:22.228080 27800 raft_consensus.cc:729] T 00000000000000000000000000000000 P f05881afb75648c89dedd2f5d476e96a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f05881afb75648c89dedd2f5d476e96a, State: Initialized, Role: FOLLOWER
I20260808 16:54:22.228225 27800 consensus_queue.cc:260] T 00000000000000000000000000000000 P f05881afb75648c89dedd2f5d476e96a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f05881afb75648c89dedd2f5d476e96a" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 45771 } }
I20260808 16:54:22.228315 27800 raft_consensus.cc:388] T 00000000000000000000000000000000 P f05881afb75648c89dedd2f5d476e96a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 16:54:22.228344 27800 raft_consensus.cc:482] T 00000000000000000000000000000000 P f05881afb75648c89dedd2f5d476e96a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 16:54:22.228408 27800 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f05881afb75648c89dedd2f5d476e96a [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:54:22.229029 27800 raft_consensus.cc:504] T 00000000000000000000000000000000 P f05881afb75648c89dedd2f5d476e96a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f05881afb75648c89dedd2f5d476e96a" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 45771 } }
I20260808 16:54:22.229140 27800 leader_election.cc:302] T 00000000000000000000000000000000 P f05881afb75648c89dedd2f5d476e96a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f05881afb75648c89dedd2f5d476e96a; no voters:
I20260808 16:54:22.229339 27800 leader_election.cc:288] T 00000000000000000000000000000000 P f05881afb75648c89dedd2f5d476e96a [CANDIDATE]: Term 1 election: Requested vote from peers
Aug 08 16:54:22 dist-test-slave-6zbq krb5kdc[26980](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208061, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260808 16:54:22.229704 27800 sys_catalog.cc:564] T 00000000000000000000000000000000 P f05881afb75648c89dedd2f5d476e96a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 16:54:22.229391 27809 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f05881afb75648c89dedd2f5d476e96a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:54:22.230283 27809 raft_consensus.cc:686] T 00000000000000000000000000000000 P f05881afb75648c89dedd2f5d476e96a [term 1 LEADER]: Becoming Leader. State: Replica: f05881afb75648c89dedd2f5d476e96a, State: Running, Role: LEADER
I20260808 16:54:22.230422 27809 consensus_queue.cc:237] T 00000000000000000000000000000000 P f05881afb75648c89dedd2f5d476e96a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f05881afb75648c89dedd2f5d476e96a" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 45771 } }
I20260808 16:54:22.230806 27809 sys_catalog.cc:455] T 00000000000000000000000000000000 P f05881afb75648c89dedd2f5d476e96a [sys.catalog]: SysCatalogTable state changed. Reason: New leader f05881afb75648c89dedd2f5d476e96a. Latest consensus state: current_term: 1 leader_uuid: "f05881afb75648c89dedd2f5d476e96a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f05881afb75648c89dedd2f5d476e96a" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 45771 } } }
I20260808 16:54:22.230952 27809 sys_catalog.cc:458] T 00000000000000000000000000000000 P f05881afb75648c89dedd2f5d476e96a [sys.catalog]: This master's current role is: LEADER
I20260808 16:54:22.231104 27800 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 16:54:22.231221 27833 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 16:54:22.234009 27811 sys_catalog.cc:455] T 00000000000000000000000000000000 P f05881afb75648c89dedd2f5d476e96a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f05881afb75648c89dedd2f5d476e96a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f05881afb75648c89dedd2f5d476e96a" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 45771 } } }
I20260808 16:54:22.234251 27811 sys_catalog.cc:458] T 00000000000000000000000000000000 P f05881afb75648c89dedd2f5d476e96a [sys.catalog]: This master's current role is: LEADER
I20260808 16:54:22.235026 27833 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 16:54:22.236878 27806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:22.223774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35388 (local address 127.8.24.62:45771)
0808 16:54:22.223956 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0808 16:54:22.223966 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 16:54:22.224001 (+ 35us) server_negotiation.cc:407] Connection header received
0808 16:54:22.224557 (+ 556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:54:22.224566 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:54:22.224627 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:54:22.224768 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:54:22.225442 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:22.226298 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:22.227106 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:22.227496 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:22.227592 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:22.229899 (+ 2307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:54:22.229903 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:54:22.229903 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:54:22.229913 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:54:22.233906 (+ 3993us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:54:22.234416 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:54:22.234419 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:54:22.234419 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:54:22.234600 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:54:22.234859 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:54:22.234860 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:54:22.234860 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:54:22.235397 (+ 537us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:54:22.235627 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0808 16:54:22.236177 (+ 550us) server_negotiation.cc:299] Negotiation successful
0808 16:54:22.236528 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":71,"threads_started":1}
I20260808 16:54:22.237964 27838 mvcc.cc:205] Tried to move back new op lower bound from 7316308222922031104 to 7316308222896386048. Current Snapshot: MvccSnapshot[applied={T|T < 7316308222922031104}]
I20260808 16:54:22.242379 27833 catalog_manager.cc:1347] Generated new cluster ID: 2097fccf84b649c098652c82087245ff
I20260808 16:54:22.242434 27833 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 16:54:22.254086 27833 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 16:54:22.254768 27833 catalog_manager.cc:1504] Loading token signing keys...
I20260808 16:54:22.260885 27833 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f05881afb75648c89dedd2f5d476e96a: Generated new TSK 0
I20260808 16:54:22.261044 27833 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.24.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.1.keytab.
Entry for principal kudu/127.8.24.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.1.keytab.
I20260808 16:54:22.971299 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.24.20:48301/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.1:0
--local_ip_for_outbound_sockets=127.8.24.1
--webserver_interface=127.8.24.1
--webserver_port=0
--tserver_master_addrs=127.8.24.62:45771
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.1.keytab
--principal=kudu/127.8.24.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:42363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:54:23.120409 27875 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:54:23.120673 27875 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:54:23.120709 27875 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:54:23.122330 27875 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:54:23.122413 27875 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:54:23.122452 27875 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.1
I20260808 16:54:23.124509 27875 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:42363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.8.24.20:48301/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.1.keytab
--principal=kudu/127.8.24.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.24.1
--webserver_port=0
--tserver_master_addrs=127.8.24.62:45771
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27875
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.1
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:54:23.124738 27875 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:54:23.125036 27875 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:54:23.131347 27885 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:54:23.131705 27882 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:54:23.132076 27883 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:54:23.132992 27875 server_base.cc:1034] running on GCE node
I20260808 16:54:23.133211 27875 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:54:23.133497 27875 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:54:23.134649 27875 hybrid_clock.cc:648] HybridClock initialized: now 1786208063134640 us; error 51 us; skew 500 ppm
Aug 08 16:54:23 dist-test-slave-6zbq krb5kdc[26980](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208063, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:54:23.137784 27875 init.cc:377] Logged in from keytab as kudu/127.8.24.1@KRBTEST.COM (short username kudu)
I20260808 16:54:23.138316 27875 webserver.cc:460] Webserver started at http://127.8.24.1:41369/ using document root <none> and password file <none>
I20260808 16:54:23.138615 27875 fs_manager.cc:359] Metadata directory not provided
I20260808 16:54:23.138685 27875 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:54:23.138833 27875 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 16:54:23 dist-test-slave-6zbq krb5kdc[26980](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208063, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.1@KRBTEST.COM for HTTP/127.8.24.20@KRBTEST.COM
I20260808 16:54:23.153373 27875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ts-0/data/instance:
uuid: "abf681eae39e4d6bac1e70b80e4cd7a9"
format_stamp: "Formatted at 2026-08-08 16:54:23 on dist-test-slave-6zbq"
server_key: "4e7bbf8c3d7aae26f7aaa01ba05b4272"
server_key_iv: "ead654d6736d3905282a1fd0cb2a383b"
server_key_version: "kuduclusterkey@0"
I20260808 16:54:23.153856 27875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ts-0/wal/instance:
uuid: "abf681eae39e4d6bac1e70b80e4cd7a9"
format_stamp: "Formatted at 2026-08-08 16:54:23 on dist-test-slave-6zbq"
server_key: "4e7bbf8c3d7aae26f7aaa01ba05b4272"
server_key_iv: "ead654d6736d3905282a1fd0cb2a383b"
server_key_version: "kuduclusterkey@0"
I20260808 16:54:23.155431 27875 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 16:54:23.168191 27894 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:54:23.168380 27875 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 16:54:23.168473 27875 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ts-0/wal
uuid: "abf681eae39e4d6bac1e70b80e4cd7a9"
format_stamp: "Formatted at 2026-08-08 16:54:23 on dist-test-slave-6zbq"
server_key: "4e7bbf8c3d7aae26f7aaa01ba05b4272"
server_key_iv: "ead654d6736d3905282a1fd0cb2a383b"
server_key_version: "kuduclusterkey@0"
I20260808 16:54:23.168556 27875 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:54:23.212261 27875 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:54:23.214629 27875 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:54:23.214821 27875 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:54:23.215113 27875 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:54:23.215521 27875 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:54:23.215608 27875 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:54:23.215700 27875 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:54:23.215778 27875 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:54:23.221833 27875 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.1:40643
I20260808 16:54:23.222257 27875 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ts-0/data/info.pb
I20260808 16:54:23.224452 28069 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.1:40643 every 8 connection(s)
I20260808 16:54:23.227023 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 27875
I20260808 16:54:23.227118 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ts-0/wal/instance
I20260808 16:54:23.227306 8288 external_mini_cluster.cc:1428] Setting key 645195a61750840cdd808a318a716858
Aug 08 16:54:23 dist-test-slave-6zbq krb5kdc[26980](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208063, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.1@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260808 16:54:23.234326 28077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:23.224984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:45771 (local address 127.8.24.1:48507)
0808 16:54:23.225184 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:23.225223 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 16:54:23.225709 (+ 486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:54:23.225967 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:54:23.225971 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:54:23.226138 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:54:23.226325 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:54:23.226329 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:54:23.227076 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:54:23.227077 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:54:23.228217 (+ 1140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:54:23.228221 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:54:23.228697 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:54:23.228698 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:54:23.228836 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:23.229295 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:54:23.229305 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:54:23.230991 (+ 1686us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:54:23.232737 (+ 1746us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:54:23.232743 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:54:23.232745 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:54:23.233006 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:54:23.233206 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:54:23.233207 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:54:23.233208 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:54:23.233363 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:54:23.233671 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:54:23.233677 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:54:23.233856 (+ 179us) client_negotiation.cc:769] Sending connection context
0808 16:54:23.233962 (+ 106us) client_negotiation.cc:240] Negotiation successful
0808 16:54:23.234111 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":80,"threads_started":1}
I20260808 16:54:23.234750 28076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:23.224999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:48507 (local address 127.8.24.62:45771)
0808 16:54:23.225124 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0808 16:54:23.225126 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:54:23.225325 (+ 199us) server_negotiation.cc:407] Connection header received
0808 16:54:23.225803 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:54:23.225804 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:54:23.225818 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:54:23.225886 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:54:23.226423 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:23.226968 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:23.228322 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:23.228567 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:23.228618 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:23.231091 (+ 2473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:54:23.231093 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:54:23.231093 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:54:23.231100 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:54:23.232650 (+ 1550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:54:23.233084 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:54:23.233085 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:54:23.233085 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:54:23.233143 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:54:23.233440 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:54:23.233441 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:54:23.233441 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:54:23.233584 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:54:23.233644 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 16:54:23.234579 (+ 935us) server_negotiation.cc:299] Negotiation successful
0808 16:54:23.234659 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":71,"threads_started":1}
I20260808 16:54:23.235203 28074 heartbeater.cc:347] Connected to a master server at 127.8.24.62:45771
I20260808 16:54:23.235311 28074 heartbeater.cc:464] Registering TS with master...
I20260808 16:54:23.235524 28074 heartbeater.cc:511] Master 127.8.24.62:45771 requested a full tablet report, sending...
I20260808 16:54:23.236032 27734 ts_manager.cc:194] Registered new tserver with Master: abf681eae39e4d6bac1e70b80e4cd7a9 (127.8.24.1:40643)
I20260808 16:54:23.236868 27734 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.1@KRBTEST.COM'} at 127.8.24.1:48507
WARNING: no policy specified for kudu/127.8.24.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.2.keytab.
Entry for principal kudu/127.8.24.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.2.keytab.
I20260808 16:54:23.284075 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.24.20:48301/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.2:0
--local_ip_for_outbound_sockets=127.8.24.2
--webserver_interface=127.8.24.2
--webserver_port=0
--tserver_master_addrs=127.8.24.62:45771
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.2.keytab
--principal=kudu/127.8.24.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:42363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 16:54:23 dist-test-slave-6zbq krb5kdc[26980](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786208063, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260808 16:54:23.410530 28086 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:54:23.410849 28086 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:54:23.410928 28086 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:54:23.412588 28086 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:54:23.412700 28086 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:54:23.412778 28086 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.2
I20260808 16:54:23.414916 28086 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:42363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.8.24.20:48301/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.2.keytab
--principal=kudu/127.8.24.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.24.2
--webserver_port=0
--tserver_master_addrs=127.8.24.62:45771
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28086
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.2
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:54:23.415424 28086 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:54:23.415714 28086 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:54:23.422091 28094 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:54:23.422112 28095 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:54:23.422106 28098 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:54:23.423024 28086 server_base.cc:1034] running on GCE node
I20260808 16:54:23.423305 28086 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:54:23.423533 28086 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:54:23.424665 28086 hybrid_clock.cc:648] HybridClock initialized: now 1786208063424648 us; error 31 us; skew 500 ppm
Aug 08 16:54:23 dist-test-slave-6zbq krb5kdc[26980](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208063, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:54:23.427801 28086 init.cc:377] Logged in from keytab as kudu/127.8.24.2@KRBTEST.COM (short username kudu)
I20260808 16:54:23.428349 28086 webserver.cc:460] Webserver started at http://127.8.24.2:36377/ using document root <none> and password file <none>
I20260808 16:54:23.428602 28086 fs_manager.cc:359] Metadata directory not provided
I20260808 16:54:23.428651 28086 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:54:23.428771 28086 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 16:54:23 dist-test-slave-6zbq krb5kdc[26980](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208063, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.2@KRBTEST.COM for HTTP/127.8.24.20@KRBTEST.COM
I20260808 16:54:23.447345 28086 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ts-1/data/instance:
uuid: "cf6a0ebb4a2440bfa0f7595dbd8dad1e"
format_stamp: "Formatted at 2026-08-08 16:54:23 on dist-test-slave-6zbq"
server_key: "ccb9cf3bf0bbc0d269bf7eaf25398211"
server_key_iv: "35bde4d2398e6278a4925da920b9779c"
server_key_version: "kuduclusterkey@0"
I20260808 16:54:23.447862 28086 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ts-1/wal/instance:
uuid: "cf6a0ebb4a2440bfa0f7595dbd8dad1e"
format_stamp: "Formatted at 2026-08-08 16:54:23 on dist-test-slave-6zbq"
server_key: "ccb9cf3bf0bbc0d269bf7eaf25398211"
server_key_iv: "35bde4d2398e6278a4925da920b9779c"
server_key_version: "kuduclusterkey@0"
I20260808 16:54:23.450127 28086 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260808 16:54:23.468839 28111 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:54:23.469056 28086 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.000s sys 0.001s
I20260808 16:54:23.469161 28086 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ts-1/wal
uuid: "cf6a0ebb4a2440bfa0f7595dbd8dad1e"
format_stamp: "Formatted at 2026-08-08 16:54:23 on dist-test-slave-6zbq"
server_key: "ccb9cf3bf0bbc0d269bf7eaf25398211"
server_key_iv: "35bde4d2398e6278a4925da920b9779c"
server_key_version: "kuduclusterkey@0"
I20260808 16:54:23.469238 28086 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:54:23.495267 28086 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:54:23.497588 28086 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:54:23.497745 28086 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:54:23.497995 28086 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:54:23.498432 28086 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:54:23.498489 28086 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:54:23.498525 28086 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:54:23.498553 28086 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:54:23.505124 28086 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.2:40639
I20260808 16:54:23.505546 28086 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ts-1/data/info.pb
I20260808 16:54:23.511410 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 28086
I20260808 16:54:23.511502 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ts-1/wal/instance
I20260808 16:54:23.511691 8288 external_mini_cluster.cc:1428] Setting key e693e511da91eaf8439554850f13a83b
I20260808 16:54:23.507309 28292 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.2:40639 every 8 connection(s)
WARNING: no policy specified for kudu/127.8.24.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.3@KRBTEST.COM" created.
Aug 08 16:54:23 dist-test-slave-6zbq krb5kdc[26980](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208063, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.2@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260808 16:54:23.529498 28076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:23.508202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:49491 (local address 127.8.24.62:45771)
0808 16:54:23.508254 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 16:54:23.508256 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:54:23.508581 (+ 325us) server_negotiation.cc:407] Connection header received
0808 16:54:23.509244 (+ 663us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:54:23.509245 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:54:23.509257 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:54:23.509292 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:54:23.509977 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:23.510577 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:23.523460 (+ 12883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:23.523780 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:23.523858 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:23.526384 (+ 2526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:54:23.526386 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:54:23.526386 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:54:23.526395 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:54:23.528001 (+ 1606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:54:23.528460 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:54:23.528461 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:54:23.528462 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:54:23.528524 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:54:23.528774 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:54:23.528775 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:54:23.528776 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:54:23.528911 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:54:23.528979 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 16:54:23.529317 (+ 338us) server_negotiation.cc:299] Negotiation successful
0808 16:54:23.529406 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 16:54:23.529652 28296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:23.507895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:45771 (local address 127.8.24.2:49491)
0808 16:54:23.508444 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:23.508489 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 16:54:23.509142 (+ 653us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:54:23.509369 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:54:23.509373 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:54:23.509602 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:54:23.509873 (+ 271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:54:23.509881 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:54:23.522431 (+ 12550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:54:23.522433 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:54:23.523363 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:54:23.523366 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:54:23.523884 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:54:23.523886 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:54:23.524011 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:23.524562 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:54:23.524574 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:54:23.526111 (+ 1537us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:54:23.528090 (+ 1979us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:54:23.528109 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:54:23.528112 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:54:23.528393 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:54:23.528581 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:54:23.528581 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:54:23.528582 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:54:23.528718 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:54:23.528986 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:54:23.528993 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:54:23.529184 (+ 191us) client_negotiation.cc:769] Sending connection context
0808 16:54:23.529297 (+ 113us) client_negotiation.cc:240] Negotiation successful
0808 16:54:23.529435 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":509,"thread_start_us":87,"threads_started":1}
I20260808 16:54:23.530155 28294 heartbeater.cc:347] Connected to a master server at 127.8.24.62:45771
I20260808 16:54:23.530246 28294 heartbeater.cc:464] Registering TS with master...
I20260808 16:54:23.530455 28294 heartbeater.cc:511] Master 127.8.24.62:45771 requested a full tablet report, sending...
I20260808 16:54:23.530830 27734 ts_manager.cc:194] Registered new tserver with Master: cf6a0ebb4a2440bfa0f7595dbd8dad1e (127.8.24.2:40639)
I20260808 16:54:23.531301 27734 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.2@KRBTEST.COM'} at 127.8.24.2:49491
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.3.keytab.
Entry for principal kudu/127.8.24.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.3.keytab.
I20260808 16:54:23.548802 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.24.20:48301/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.3:0
--local_ip_for_outbound_sockets=127.8.24.3
--webserver_interface=127.8.24.3
--webserver_port=0
--tserver_master_addrs=127.8.24.62:45771
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.3.keytab
--principal=kudu/127.8.24.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:42363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:54:23.702252 28303 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:54:23.702549 28303 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:54:23.702584 28303 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:54:23.704195 28303 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:54:23.704248 28303 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:54:23.704283 28303 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.3
I20260808 16:54:23.706336 28303 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:42363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.8.24.20:48301/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.3.keytab
--principal=kudu/127.8.24.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.24.3
--webserver_port=0
--tserver_master_addrs=127.8.24.62:45771
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28303
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.3
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:54:23.706593 28303 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:54:23.706827 28303 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:54:23.712934 28313 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:54:23.713085 28310 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:54:23.713243 28309 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:54:23.714095 28303 server_base.cc:1034] running on GCE node
I20260808 16:54:23.714372 28303 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:54:23.714615 28303 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:54:23.716696 28303 hybrid_clock.cc:648] HybridClock initialized: now 1786208063716676 us; error 32 us; skew 500 ppm
Aug 08 16:54:23 dist-test-slave-6zbq krb5kdc[26980](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208063, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:54:23.719727 28303 init.cc:377] Logged in from keytab as kudu/127.8.24.3@KRBTEST.COM (short username kudu)
I20260808 16:54:23.720260 28303 webserver.cc:460] Webserver started at http://127.8.24.3:40271/ using document root <none> and password file <none>
I20260808 16:54:23.720480 28303 fs_manager.cc:359] Metadata directory not provided
I20260808 16:54:23.720525 28303 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:54:23.720640 28303 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 16:54:23 dist-test-slave-6zbq krb5kdc[26980](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208063, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.3@KRBTEST.COM for HTTP/127.8.24.20@KRBTEST.COM
I20260808 16:54:23.733944 28303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ts-2/data/instance:
uuid: "1334561c8ce74535b255a792168ea9d4"
format_stamp: "Formatted at 2026-08-08 16:54:23 on dist-test-slave-6zbq"
server_key: "0b0d392769d47ac054e46d8be3fe6dad"
server_key_iv: "3e23a314c8267949c6208bfafb8e17a7"
server_key_version: "kuduclusterkey@0"
I20260808 16:54:23.734455 28303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ts-2/wal/instance:
uuid: "1334561c8ce74535b255a792168ea9d4"
format_stamp: "Formatted at 2026-08-08 16:54:23 on dist-test-slave-6zbq"
server_key: "0b0d392769d47ac054e46d8be3fe6dad"
server_key_iv: "3e23a314c8267949c6208bfafb8e17a7"
server_key_version: "kuduclusterkey@0"
I20260808 16:54:23.736896 28303 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260808 16:54:23.752921 28322 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:54:23.753083 28303 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 16:54:23.753175 28303 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ts-2/wal
uuid: "1334561c8ce74535b255a792168ea9d4"
format_stamp: "Formatted at 2026-08-08 16:54:23 on dist-test-slave-6zbq"
server_key: "0b0d392769d47ac054e46d8be3fe6dad"
server_key_iv: "3e23a314c8267949c6208bfafb8e17a7"
server_key_version: "kuduclusterkey@0"
I20260808 16:54:23.753264 28303 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:54:23.793042 28303 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:54:23.795359 28303 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:54:23.795507 28303 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:54:23.795720 28303 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:54:23.796093 28303 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:54:23.796160 28303 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:54:23.796196 28303 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:54:23.796217 28303 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:54:23.802935 28303 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.3:41989
I20260808 16:54:23.803370 28303 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ts-2/data/info.pb
I20260808 16:54:23.809892 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 28303
I20260808 16:54:23.809989 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ts-2/wal/instance
I20260808 16:54:23.810226 8288 external_mini_cluster.cc:1428] Setting key 2127130d43fe50ea7ece47a1c9d44787
I20260808 16:54:23.814713 28499 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.3:41989 every 8 connection(s)
Aug 08 16:54:23 dist-test-slave-6zbq krb5kdc[26980](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208063, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.3@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260808 16:54:23.824924 28076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:23.815317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:46103 (local address 127.8.24.62:45771)
0808 16:54:23.815542 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0808 16:54:23.815544 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:54:23.815633 (+ 89us) server_negotiation.cc:407] Connection header received
0808 16:54:23.816273 (+ 640us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:54:23.816274 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:54:23.816285 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:54:23.816345 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:54:23.817043 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:23.817770 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:23.819198 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:23.819516 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:23.819587 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:23.821962 (+ 2375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:54:23.821964 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:54:23.821964 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:54:23.821971 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:54:23.823502 (+ 1531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:54:23.823954 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:54:23.823955 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:54:23.823956 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:54:23.824010 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:54:23.824281 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:54:23.824282 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:54:23.824282 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:54:23.824399 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:54:23.824456 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 16:54:23.824758 (+ 302us) server_negotiation.cc:299] Negotiation successful
0808 16:54:23.824832 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199}
I20260808 16:54:23.825106 28503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:23.815319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:45771 (local address 127.8.24.3:46103)
0808 16:54:23.815497 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:23.815547 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 16:54:23.816179 (+ 632us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:54:23.816424 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:54:23.816428 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:54:23.816664 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:54:23.816928 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:54:23.816934 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:54:23.817887 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:54:23.817888 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:54:23.819103 (+ 1215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:54:23.819106 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:54:23.819625 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:54:23.819627 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:54:23.819765 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:23.820260 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:54:23.820269 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:54:23.821861 (+ 1592us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:54:23.823585 (+ 1724us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:54:23.823592 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:54:23.823594 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:54:23.823877 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:54:23.824069 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:54:23.824070 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:54:23.824071 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:54:23.824217 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:54:23.824458 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:54:23.824463 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:54:23.824624 (+ 161us) client_negotiation.cc:769] Sending connection context
0808 16:54:23.824738 (+ 114us) client_negotiation.cc:240] Negotiation successful
0808 16:54:23.824886 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":86,"threads_started":1}
I20260808 16:54:23.825673 28500 heartbeater.cc:347] Connected to a master server at 127.8.24.62:45771
I20260808 16:54:23.825769 28500 heartbeater.cc:464] Registering TS with master...
I20260808 16:54:23.825946 28500 heartbeater.cc:511] Master 127.8.24.62:45771 requested a full tablet report, sending...
I20260808 16:54:23.826316 27734 ts_manager.cc:194] Registered new tserver with Master: 1334561c8ce74535b255a792168ea9d4 (127.8.24.3:41989)
I20260808 16:54:23.826800 27734 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.3@KRBTEST.COM'} at 127.8.24.3:46103
I20260808 16:54:23.832648 8288 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 16:54:23.833400 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 27875
I20260808 16:54:23.862550 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 28086
I20260808 16:54:23.869241 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 28303
I20260808 16:54:23.890625 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 27682
I20260808 16:54:23.899791 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:45771
--webserver_interface=127.8.24.62
--webserver_port=44367
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:42363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:45771
--ranger_config_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.24.20:48301/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.8.24.20:48301/kms with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:54:23.977694 28506 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:54:23.977862 28506 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:54:23.977881 28506 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:54:23.978926 28506 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 16:54:23.978960 28506 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:54:23.978971 28506 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:54:23.978991 28506 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 16:54:23.979001 28506 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 16:54:23.980438 28506 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:42363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.8.24.20:48301/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:45771
--ranger_config_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:45771
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.24.62
--webserver_port=44367
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28506
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:54:23.980600 28506 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:54:23.980749 28506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:54:23.984949 28517 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:54:23.984959 28521 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:54:23.985004 28524 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:54:23.985176 28506 server_base.cc:1034] running on GCE node
I20260808 16:54:23.985359 28506 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:54:23.985586 28506 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:54:23.986706 28506 hybrid_clock.cc:648] HybridClock initialized: now 1786208063986696 us; error 27 us; skew 500 ppm
Aug 08 16:54:23 dist-test-slave-6zbq krb5kdc[26980](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208063, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:54:23.988909 28506 init.cc:377] Logged in from keytab as kudu/127.8.24.62@KRBTEST.COM (short username kudu)
I20260808 16:54:23.989296 28506 webserver.cc:460] Webserver started at http://127.8.24.62:44367/ using document root <none> and password file <none>
I20260808 16:54:23.989467 28506 fs_manager.cc:359] Metadata directory not provided
I20260808 16:54:23.989511 28506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 16:54:24 dist-test-slave-6zbq krb5kdc[26980](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208063, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for HTTP/127.8.24.20@KRBTEST.COM
I20260808 16:54:24.064177 28506 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 16:54:24.065042 28541 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:54:24.065225 28506 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 16:54:24.065325 28506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/master-0/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/master-0/wal
uuid: "f05881afb75648c89dedd2f5d476e96a"
format_stamp: "Formatted at 2026-08-08 16:54:22 on dist-test-slave-6zbq"
server_key: "2ab9da9c65ba76a224d733803a51cc88"
server_key_iv: "f3b19bc49d74a9075558a7ae2d1dfac6"
server_key_version: "kuduclusterkey@0"
I20260808 16:54:24.065744 28506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:54:24.080149 28506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:54:24.081880 28506 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:54:24.081981 28506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:54:24.085703 28506 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.62:45771
I20260808 16:54:24.085754 28624 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.62:45771 every 8 connection(s)
I20260808 16:54:24.086073 28506 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
I20260808 16:54:24.087214 28625 sys_catalog.cc:263] Verifying existing consensus state
I20260808 16:54:24.087999 28625 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f05881afb75648c89dedd2f5d476e96a: Bootstrap starting.
I20260808 16:54:24.095258 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 28506
I20260808 16:54:24.095577 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.24.20:48301/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.1:40643
--local_ip_for_outbound_sockets=127.8.24.1
--tserver_master_addrs=127.8.24.62:45771
--webserver_port=41369
--webserver_interface=127.8.24.1
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.1.keytab
--principal=kudu/127.8.24.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:42363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.8.24.20:48301/kms with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 16:54:24.096048 28625 log.cc:824] T 00000000000000000000000000000000 P f05881afb75648c89dedd2f5d476e96a: Log is configured to *not* fsync() on all Append() calls
I20260808 16:54:24.097807 28625 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f05881afb75648c89dedd2f5d476e96a: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 16:54:24.098042 28625 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f05881afb75648c89dedd2f5d476e96a: Bootstrap complete.
I20260808 16:54:24.100317 28625 raft_consensus.cc:348] T 00000000000000000000000000000000 P f05881afb75648c89dedd2f5d476e96a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f05881afb75648c89dedd2f5d476e96a" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 45771 } }
I20260808 16:54:24.100538 28625 raft_consensus.cc:729] T 00000000000000000000000000000000 P f05881afb75648c89dedd2f5d476e96a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f05881afb75648c89dedd2f5d476e96a, State: Initialized, Role: FOLLOWER
I20260808 16:54:24.100677 28625 consensus_queue.cc:260] T 00000000000000000000000000000000 P f05881afb75648c89dedd2f5d476e96a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f05881afb75648c89dedd2f5d476e96a" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 45771 } }
I20260808 16:54:24.100745 28625 raft_consensus.cc:388] T 00000000000000000000000000000000 P f05881afb75648c89dedd2f5d476e96a [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 16:54:24.100775 28625 raft_consensus.cc:482] T 00000000000000000000000000000000 P f05881afb75648c89dedd2f5d476e96a [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 16:54:24.100819 28625 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f05881afb75648c89dedd2f5d476e96a [term 1 FOLLOWER]: Advancing to term 2
I20260808 16:54:24.101770 28625 raft_consensus.cc:504] T 00000000000000000000000000000000 P f05881afb75648c89dedd2f5d476e96a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f05881afb75648c89dedd2f5d476e96a" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 45771 } }
I20260808 16:54:24.101858 28625 leader_election.cc:302] T 00000000000000000000000000000000 P f05881afb75648c89dedd2f5d476e96a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f05881afb75648c89dedd2f5d476e96a; no voters:
I20260808 16:54:24.102006 28625 leader_election.cc:288] T 00000000000000000000000000000000 P f05881afb75648c89dedd2f5d476e96a [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 16:54:24.102054 28630 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f05881afb75648c89dedd2f5d476e96a [term 2 FOLLOWER]: Leader election won for term 2
I20260808 16:54:24.102164 28630 raft_consensus.cc:686] T 00000000000000000000000000000000 P f05881afb75648c89dedd2f5d476e96a [term 2 LEADER]: Becoming Leader. State: Replica: f05881afb75648c89dedd2f5d476e96a, State: Running, Role: LEADER
I20260808 16:54:24.102275 28625 sys_catalog.cc:564] T 00000000000000000000000000000000 P f05881afb75648c89dedd2f5d476e96a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 16:54:24.102255 28630 consensus_queue.cc:237] T 00000000000000000000000000000000 P f05881afb75648c89dedd2f5d476e96a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f05881afb75648c89dedd2f5d476e96a" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 45771 } }
I20260808 16:54:24.102582 28630 sys_catalog.cc:455] T 00000000000000000000000000000000 P f05881afb75648c89dedd2f5d476e96a [sys.catalog]: SysCatalogTable state changed. Reason: New leader f05881afb75648c89dedd2f5d476e96a. Latest consensus state: current_term: 2 leader_uuid: "f05881afb75648c89dedd2f5d476e96a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f05881afb75648c89dedd2f5d476e96a" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 45771 } } }
I20260808 16:54:24.102679 28630 sys_catalog.cc:458] T 00000000000000000000000000000000 P f05881afb75648c89dedd2f5d476e96a [sys.catalog]: This master's current role is: LEADER
I20260808 16:54:24.102579 28633 sys_catalog.cc:455] T 00000000000000000000000000000000 P f05881afb75648c89dedd2f5d476e96a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "f05881afb75648c89dedd2f5d476e96a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f05881afb75648c89dedd2f5d476e96a" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 45771 } } }
I20260808 16:54:24.102857 28633 sys_catalog.cc:458] T 00000000000000000000000000000000 P f05881afb75648c89dedd2f5d476e96a [sys.catalog]: This master's current role is: LEADER
I20260808 16:54:24.102974 28639 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 16:54:24.103444 28639 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 16:54:24.103576 28639 catalog_manager.cc:1259] Loaded cluster ID: 2097fccf84b649c098652c82087245ff
I20260808 16:54:24.103626 28639 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 16:54:24.103771 28625 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 16:54:24.107048 28639 catalog_manager.cc:1504] Loading token signing keys...
I20260808 16:54:24.107577 28639 catalog_manager.cc:5879] T 00000000000000000000000000000000 P f05881afb75648c89dedd2f5d476e96a: Loaded TSK: 0
I20260808 16:54:24.107903 28639 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 16:54:24.179905 28629 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:54:24.180140 28629 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:54:24.180174 28629 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:54:24.181818 28629 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:54:24.181864 28629 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:54:24.181898 28629 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.1
I20260808 16:54:24.183607 28629 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:42363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.8.24.20:48301/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.1.keytab
--principal=kudu/127.8.24.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.1:40643
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.24.1
--webserver_port=41369
--tserver_master_addrs=127.8.24.62:45771
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28629
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.1
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:54:24.183821 28629 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:54:24.184024 28629 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:54:24.188704 28687 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:54:24.188753 28682 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:54:24.189090 28629 server_base.cc:1034] running on GCE node
W20260808 16:54:24.189246 28685 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:54:24.189442 28629 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:54:24.189666 28629 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:54:24.190840 28629 hybrid_clock.cc:648] HybridClock initialized: now 1786208064190827 us; error 27 us; skew 500 ppm
Aug 08 16:54:24 dist-test-slave-6zbq krb5kdc[26980](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208064, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:54:24.193234 28629 init.cc:377] Logged in from keytab as kudu/127.8.24.1@KRBTEST.COM (short username kudu)
I20260808 16:54:24.193629 28629 webserver.cc:460] Webserver started at http://127.8.24.1:41369/ using document root <none> and password file <none>
I20260808 16:54:24.193835 28629 fs_manager.cc:359] Metadata directory not provided
I20260808 16:54:24.193895 28629 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 16:54:24 dist-test-slave-6zbq krb5kdc[26980](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208064, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.1@KRBTEST.COM for HTTP/127.8.24.20@KRBTEST.COM
I20260808 16:54:24.218743 28629 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.000s
I20260808 16:54:24.221534 28699 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:54:24.221691 28629 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 16:54:24.221774 28629 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ts-0/wal
uuid: "abf681eae39e4d6bac1e70b80e4cd7a9"
format_stamp: "Formatted at 2026-08-08 16:54:23 on dist-test-slave-6zbq"
server_key: "4e7bbf8c3d7aae26f7aaa01ba05b4272"
server_key_iv: "ead654d6736d3905282a1fd0cb2a383b"
server_key_version: "kuduclusterkey@0"
I20260808 16:54:24.222208 28629 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:54:24.241346 28629 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:54:24.243371 28629 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:54:24.243520 28629 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:54:24.243731 28629 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:54:24.244071 28629 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:54:24.244131 28629 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:54:24.244163 28629 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:54:24.244195 28629 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:54:24.250368 28629 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.1:40643
I20260808 16:54:24.250725 28629 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ts-0/data/info.pb
I20260808 16:54:24.251389 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 28629
I20260808 16:54:24.251662 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.24.20:48301/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.2:40639
--local_ip_for_outbound_sockets=127.8.24.2
--tserver_master_addrs=127.8.24.62:45771
--webserver_port=36377
--webserver_interface=127.8.24.2
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.2.keytab
--principal=kudu/127.8.24.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:42363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.8.24.20:48301/kms with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 16:54:24 dist-test-slave-6zbq krb5kdc[26980](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208064, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.1@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260808 16:54:24.263794 28891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:24.251749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:47485 (local address 127.8.24.62:45771)
0808 16:54:24.252152 (+ 403us) server_negotiation.cc:206] Beginning negotiation
0808 16:54:24.252162 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 16:54:24.252985 (+ 823us) server_negotiation.cc:407] Connection header received
0808 16:54:24.253138 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:54:24.253145 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:54:24.253203 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:54:24.253328 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:54:24.254449 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:24.255335 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:24.256953 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:24.257347 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:24.257449 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:24.259325 (+ 1876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:54:24.259329 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:54:24.259330 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:54:24.259342 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:54:24.261661 (+ 2319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:54:24.262114 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:54:24.262117 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:54:24.262117 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:54:24.262226 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:54:24.262546 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:54:24.262548 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:54:24.262548 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:54:24.262949 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:54:24.263012 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 16:54:24.263419 (+ 407us) server_negotiation.cc:299] Negotiation successful
0808 16:54:24.263541 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":78,"threads_started":1}
I20260808 16:54:24.265763 28890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:24.251356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:45771 (local address 127.8.24.1:47485)
0808 16:54:24.252240 (+ 884us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:24.252280 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 16:54:24.252841 (+ 561us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:54:24.253477 (+ 636us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:54:24.253480 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:54:24.253640 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:54:24.254201 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:54:24.254206 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:54:24.255749 (+ 1543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:54:24.255750 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:54:24.256873 (+ 1123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:54:24.256876 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:54:24.257440 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:54:24.257442 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:54:24.257557 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:24.257965 (+ 408us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:54:24.257973 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:54:24.259162 (+ 1189us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:54:24.261751 (+ 2589us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:54:24.261757 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:54:24.261759 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:54:24.262027 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:54:24.262302 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:54:24.262304 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:54:24.262304 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:54:24.262453 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:54:24.263064 (+ 611us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:54:24.263070 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:54:24.263224 (+ 154us) client_negotiation.cc:769] Sending connection context
0808 16:54:24.264198 (+ 974us) client_negotiation.cc:240] Negotiation successful
0808 16:54:24.264349 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":845,"thread_start_us":73,"threads_started":1}
I20260808 16:54:24.266772 28885 heartbeater.cc:347] Connected to a master server at 127.8.24.62:45771
I20260808 16:54:24.266852 28885 heartbeater.cc:464] Registering TS with master...
I20260808 16:54:24.267014 28885 heartbeater.cc:511] Master 127.8.24.62:45771 requested a full tablet report, sending...
W20260808 16:54:24.267902 28885 heartbeater.cc:644] Failed to heartbeat to 127.8.24.62:45771 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 16:54:24.267967 28885 heartbeater.cc:464] Registering TS with master...
I20260808 16:54:24.268023 28885 heartbeater.cc:511] Master 127.8.24.62:45771 requested a full tablet report, sending...
I20260808 16:54:24.268441 28885 heartbeater.cc:464] Registering TS with master...
I20260808 16:54:24.268515 28885 heartbeater.cc:511] Master 127.8.24.62:45771 requested a full tablet report, sending...
W20260808 16:54:24.268939 28885 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 16:54:24.270514 28884 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.1:40643 every 8 connection(s)
W20260808 16:54:24.369025 28892 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:54:24.369293 28892 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:54:24.369326 28892 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:54:24.370862 28892 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:54:24.370919 28892 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:54:24.370956 28892 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.2
I20260808 16:54:24.373056 28892 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:42363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.8.24.20:48301/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.2.keytab
--principal=kudu/127.8.24.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.2:40639
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.24.2
--webserver_port=36377
--tserver_master_addrs=127.8.24.62:45771
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28892
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.2
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:54:24.373279 28892 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:54:24.373502 28892 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:54:24.379637 28904 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:54:24.379647 28901 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:54:24.380270 28892 server_base.cc:1034] running on GCE node
W20260808 16:54:24.380442 28900 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:54:24.380694 28892 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:54:24.380939 28892 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:54:24.382121 28892 hybrid_clock.cc:648] HybridClock initialized: now 1786208064382097 us; error 36 us; skew 500 ppm
Aug 08 16:54:24 dist-test-slave-6zbq krb5kdc[26980](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208064, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:54:24.385061 28892 init.cc:377] Logged in from keytab as kudu/127.8.24.2@KRBTEST.COM (short username kudu)
I20260808 16:54:24.385525 28892 webserver.cc:460] Webserver started at http://127.8.24.2:36377/ using document root <none> and password file <none>
I20260808 16:54:24.385774 28892 fs_manager.cc:359] Metadata directory not provided
I20260808 16:54:24.385836 28892 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 16:54:24 dist-test-slave-6zbq krb5kdc[26980](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208064, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.2@KRBTEST.COM for HTTP/127.8.24.20@KRBTEST.COM
I20260808 16:54:24.407763 28892 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 16:54:24.409525 28919 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:54:24.409693 28892 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 16:54:24.409767 28892 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ts-1/wal
uuid: "cf6a0ebb4a2440bfa0f7595dbd8dad1e"
format_stamp: "Formatted at 2026-08-08 16:54:23 on dist-test-slave-6zbq"
server_key: "ccb9cf3bf0bbc0d269bf7eaf25398211"
server_key_iv: "35bde4d2398e6278a4925da920b9779c"
server_key_version: "kuduclusterkey@0"
I20260808 16:54:24.410189 28892 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:54:24.433790 28892 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:54:24.435911 28892 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:54:24.436043 28892 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:54:24.436254 28892 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:54:24.436571 28892 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:54:24.436622 28892 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:54:24.436648 28892 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:54:24.436671 28892 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:54:24.443884 28892 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.2:40639
I20260808 16:54:24.443918 29096 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.2:40639 every 8 connection(s)
I20260808 16:54:24.444324 28892 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ts-1/data/info.pb
Aug 08 16:54:24 dist-test-slave-6zbq krb5kdc[26980](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208064, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.2@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260808 16:54:24.452072 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 28892
I20260808 16:54:24.452355 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.24.20:48301/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.3:41989
--local_ip_for_outbound_sockets=127.8.24.3
--tserver_master_addrs=127.8.24.62:45771
--webserver_port=40271
--webserver_interface=127.8.24.3
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.3.keytab
--principal=kudu/127.8.24.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:42363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.8.24.20:48301/kms with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 16:54:24.453707 28891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:24.445809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:47109 (local address 127.8.24.62:45771)
0808 16:54:24.445852 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 16:54:24.445854 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:54:24.445868 (+ 14us) server_negotiation.cc:407] Connection header received
0808 16:54:24.445912 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:54:24.445912 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:54:24.445923 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:54:24.445962 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:54:24.446628 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:24.447101 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:24.448496 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:24.448674 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:24.448723 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:24.450872 (+ 2149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:54:24.450874 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:54:24.450874 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:54:24.450882 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:54:24.452191 (+ 1309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:54:24.452657 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:54:24.452658 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:54:24.452658 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:54:24.452732 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:54:24.453028 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:54:24.453029 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:54:24.453029 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:54:24.453152 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:54:24.453214 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 16:54:24.453531 (+ 317us) server_negotiation.cc:299] Negotiation successful
0808 16:54:24.453614 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 16:54:24.453876 29104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:24.444991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:45771 (local address 127.8.24.2:47109)
0808 16:54:24.445169 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:24.445214 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 16:54:24.445812 (+ 598us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:54:24.446040 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:54:24.446044 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:54:24.446249 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:54:24.446535 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:54:24.446543 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:54:24.447191 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:54:24.447192 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:54:24.448420 (+ 1228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:54:24.448423 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:54:24.448773 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:54:24.448775 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:54:24.448912 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:24.449340 (+ 428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:54:24.449350 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:54:24.450781 (+ 1431us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:54:24.452287 (+ 1506us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:54:24.452293 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:54:24.452295 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:54:24.452571 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:54:24.452818 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:54:24.452818 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:54:24.452819 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:54:24.452956 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:54:24.453225 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:54:24.453232 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:54:24.453383 (+ 151us) client_negotiation.cc:769] Sending connection context
0808 16:54:24.453506 (+ 123us) client_negotiation.cc:240] Negotiation successful
0808 16:54:24.453663 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":83,"threads_started":1}
I20260808 16:54:24.454531 29098 heartbeater.cc:347] Connected to a master server at 127.8.24.62:45771
I20260808 16:54:24.454623 29098 heartbeater.cc:464] Registering TS with master...
I20260808 16:54:24.454799 29098 heartbeater.cc:511] Master 127.8.24.62:45771 requested a full tablet report, sending...
W20260808 16:54:24.455478 29098 heartbeater.cc:644] Failed to heartbeat to 127.8.24.62:45771 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 16:54:24.455547 29098 heartbeater.cc:464] Registering TS with master...
I20260808 16:54:24.455606 29098 heartbeater.cc:511] Master 127.8.24.62:45771 requested a full tablet report, sending...
I20260808 16:54:24.455950 29098 heartbeater.cc:464] Registering TS with master...
I20260808 16:54:24.456068 29098 heartbeater.cc:511] Master 127.8.24.62:45771 requested a full tablet report, sending...
W20260808 16:54:24.456411 29098 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 16:54:24.553684 29106 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:54:24.553936 29106 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:54:24.553968 29106 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:54:24.555646 29106 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:54:24.555699 29106 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:54:24.555738 29106 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.3
I20260808 16:54:24.557883 29106 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:42363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.8.24.20:48301/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.3.keytab
--principal=kudu/127.8.24.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.3:41989
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.24.3
--webserver_port=40271
--tserver_master_addrs=127.8.24.62:45771
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29106
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.3
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:54:24.558130 29106 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:54:24.558393 29106 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:54:24.564433 29113 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:54:24.564452 29117 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:54:24.564792 29119 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:54:24.565258 29106 server_base.cc:1034] running on GCE node
I20260808 16:54:24.565454 29106 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:54:24.565702 29106 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:54:24.566850 29106 hybrid_clock.cc:648] HybridClock initialized: now 1786208064566824 us; error 40 us; skew 500 ppm
Aug 08 16:54:24 dist-test-slave-6zbq krb5kdc[26980](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208064, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:54:24.569880 29106 init.cc:377] Logged in from keytab as kudu/127.8.24.3@KRBTEST.COM (short username kudu)
I20260808 16:54:24.570389 29106 webserver.cc:460] Webserver started at http://127.8.24.3:40271/ using document root <none> and password file <none>
I20260808 16:54:24.570655 29106 fs_manager.cc:359] Metadata directory not provided
I20260808 16:54:24.570730 29106 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 16:54:24 dist-test-slave-6zbq krb5kdc[26980](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208064, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.3@KRBTEST.COM for HTTP/127.8.24.20@KRBTEST.COM
I20260808 16:54:24.597759 29106 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.000s
I20260808 16:54:24.599046 29133 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:54:24.599211 29106 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 16:54:24.599299 29106 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ts-2/wal
uuid: "1334561c8ce74535b255a792168ea9d4"
format_stamp: "Formatted at 2026-08-08 16:54:23 on dist-test-slave-6zbq"
server_key: "0b0d392769d47ac054e46d8be3fe6dad"
server_key_iv: "3e23a314c8267949c6208bfafb8e17a7"
server_key_version: "kuduclusterkey@0"
I20260808 16:54:24.599637 29106 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:54:24.620414 29106 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:54:24.622063 29106 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:54:24.622189 29106 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:54:24.622390 29106 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:54:24.622676 29106 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:54:24.622710 29106 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:54:24.622738 29106 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:54:24.622761 29106 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:54:24.629274 29106 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.3:41989
I20260808 16:54:24.629386 29319 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.3:41989 every 8 connection(s)
I20260808 16:54:24.629663 29106 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ts-2/data/info.pb
Aug 08 16:54:24 dist-test-slave-6zbq krb5kdc[26980](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208064, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.3@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260808 16:54:24.639413 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 29106
I20260808 16:54:24.640097 28891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:24.630289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:44401 (local address 127.8.24.62:45771)
0808 16:54:24.630333 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 16:54:24.630335 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:54:24.630566 (+ 231us) server_negotiation.cc:407] Connection header received
0808 16:54:24.631033 (+ 467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:54:24.631035 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:54:24.631049 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:54:24.631097 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:54:24.631646 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:24.632381 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:24.633668 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:24.633968 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:24.634015 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:24.636437 (+ 2422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:54:24.636439 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:54:24.636439 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:54:24.636446 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:54:24.638005 (+ 1559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:54:24.638511 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:54:24.638512 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:54:24.638512 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:54:24.638569 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:54:24.638837 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:54:24.638839 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:54:24.638839 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:54:24.638966 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:54:24.639016 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 16:54:24.639937 (+ 921us) server_negotiation.cc:299] Negotiation successful
0808 16:54:24.640015 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 16:54:24.640518 29329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:24.630295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:45771 (local address 127.8.24.3:44401)
0808 16:54:24.630459 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:24.630497 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 16:54:24.630940 (+ 443us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:54:24.631170 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:54:24.631175 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:54:24.631328 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:54:24.631530 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:54:24.631537 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:54:24.632500 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:54:24.632501 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:54:24.633560 (+ 1059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:54:24.633563 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:54:24.634147 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:54:24.634148 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:54:24.634290 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:24.634784 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:54:24.634794 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:54:24.636327 (+ 1533us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:54:24.638101 (+ 1774us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:54:24.638107 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:54:24.638109 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:54:24.638426 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:54:24.638633 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:54:24.638633 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:54:24.638634 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:54:24.638760 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:54:24.639051 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:54:24.639056 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:54:24.639215 (+ 159us) client_negotiation.cc:769] Sending connection context
0808 16:54:24.640219 (+ 1004us) client_negotiation.cc:240] Negotiation successful
0808 16:54:24.640367 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":101,"threads_started":1}
I20260808 16:54:24.640924 29325 heartbeater.cc:347] Connected to a master server at 127.8.24.62:45771
I20260808 16:54:24.641014 29325 heartbeater.cc:464] Registering TS with master...
I20260808 16:54:24.641185 29325 heartbeater.cc:511] Master 127.8.24.62:45771 requested a full tablet report, sending...
W20260808 16:54:24.641752 29325 heartbeater.cc:644] Failed to heartbeat to 127.8.24.62:45771 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 16:54:24.641824 29325 heartbeater.cc:464] Registering TS with master...
I20260808 16:54:24.641888 29325 heartbeater.cc:511] Master 127.8.24.62:45771 requested a full tablet report, sending...
I20260808 16:54:24.642230 29325 heartbeater.cc:464] Registering TS with master...
I20260808 16:54:24.642305 29325 heartbeater.cc:511] Master 127.8.24.62:45771 requested a full tablet report, sending...
W20260808 16:54:24.642654 29325 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 16:54:24.650101 29331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:24.639786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35398 (local address 127.8.24.62:45771)
0808 16:54:24.639909 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0808 16:54:24.639911 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:54:24.640101 (+ 190us) server_negotiation.cc:407] Connection header received
0808 16:54:24.640122 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:54:24.640123 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:54:24.640134 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:54:24.640191 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:54:24.640965 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:24.641711 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:24.643007 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:24.643297 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:24.643373 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:24.644875 (+ 1502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:54:24.644877 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:54:24.644877 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:54:24.644884 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:54:24.647476 (+ 2592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:54:24.647802 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:54:24.647803 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:54:24.647803 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:54:24.647869 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:54:24.649081 (+ 1212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:54:24.649082 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:54:24.649083 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:54:24.649217 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:54:24.649384 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0808 16:54:24.649426 (+ 42us) server_negotiation.cc:299] Negotiation successful
0808 16:54:24.650004 (+ 578us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93,"thread_start_us":58,"threads_started":1}
Aug 08 16:54:25 dist-test-slave-6zbq krb5kdc[26980](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786208065, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:54:25.269125 28885 heartbeater.cc:464] Registering TS with master...
I20260808 16:54:25.269265 28885 heartbeater.cc:511] Master 127.8.24.62:45771 requested a full tablet report, sending...
I20260808 16:54:25.269614 28563 ts_manager.cc:194] Registered new tserver with Master: abf681eae39e4d6bac1e70b80e4cd7a9 (127.8.24.1:40643)
I20260808 16:54:25.270697 28563 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.1@KRBTEST.COM'} at 127.8.24.1:47485
Aug 08 16:54:25 dist-test-slave-6zbq krb5kdc[26980](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208065, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for HTTP/127.8.24.20@KRBTEST.COM
I20260808 16:54:25.456632 29098 heartbeater.cc:464] Registering TS with master...
I20260808 16:54:25.456763 29098 heartbeater.cc:511] Master 127.8.24.62:45771 requested a full tablet report, sending...
I20260808 16:54:25.457029 28563 ts_manager.cc:194] Registered new tserver with Master: cf6a0ebb4a2440bfa0f7595dbd8dad1e (127.8.24.2:40639)
I20260808 16:54:25.457552 28563 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.2@KRBTEST.COM'} at 127.8.24.2:47109
I20260808 16:54:25.642808 29325 heartbeater.cc:464] Registering TS with master...
I20260808 16:54:25.642946 29325 heartbeater.cc:511] Master 127.8.24.62:45771 requested a full tablet report, sending...
I20260808 16:54:25.643226 28563 ts_manager.cc:194] Registered new tserver with Master: 1334561c8ce74535b255a792168ea9d4 (127.8.24.3:41989)
I20260808 16:54:25.643725 28563 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.3@KRBTEST.COM'} at 127.8.24.3:44401
I20260808 16:54:25.647802 8288 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 16:54:25.658222 29354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:25.648813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35410 (local address 127.8.24.62:45771)
0808 16:54:25.649071 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0808 16:54:25.649073 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:54:25.649176 (+ 103us) server_negotiation.cc:407] Connection header received
0808 16:54:25.649276 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:54:25.649277 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:54:25.649289 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:54:25.649331 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:54:25.650071 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:25.650839 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:25.651863 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:25.652193 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:25.652260 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:25.653740 (+ 1480us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:54:25.653741 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:54:25.653742 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:54:25.653749 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:54:25.656970 (+ 3221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:54:25.657361 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:54:25.657363 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:54:25.657363 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:54:25.657435 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:54:25.657663 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:54:25.657664 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:54:25.657664 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:54:25.657803 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:54:25.657902 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 16:54:25.658041 (+ 139us) server_negotiation.cc:299] Negotiation successful
0808 16:54:25.658126 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":235,"thread_start_us":63,"threads_started":1}
I20260808 16:54:25.660140 28563 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35410:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 16:54:25.662299 28563 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 16:54:25.676491 29371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:25.669928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.2:40639 (local address 127.0.0.1:39520)
0808 16:54:25.671958 (+ 2030us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:25.671965 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0808 16:54:25.672006 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:54:25.672321 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:54:25.672324 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:54:25.672327 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:54:25.672389 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:54:25.672391 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:54:25.673784 (+ 1393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:54:25.673785 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:54:25.675128 (+ 1343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:54:25.675131 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:54:25.676181 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:54:25.676182 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:54:25.676266 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:25.676281 (+ 15us) client_negotiation.cc:769] Sending connection context
0808 16:54:25.676345 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 16:54:25.676398 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2018,"mutex_wait_us":32,"thread_start_us":53,"threads_started":1}
I20260808 16:54:25.676595 29369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:25.669777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:40643 (local address 127.0.0.1:52440)
0808 16:54:25.671834 (+ 2057us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:25.671842 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 16:54:25.671894 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:54:25.672401 (+ 507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:54:25.672402 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:54:25.672404 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:54:25.672446 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:54:25.672448 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:54:25.673740 (+ 1292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:54:25.673741 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:54:25.675374 (+ 1633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:54:25.675377 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:54:25.676370 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:54:25.676371 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:54:25.676434 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:25.676438 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 16:54:25.676483 (+ 45us) client_negotiation.cc:240] Negotiation successful
0808 16:54:25.676519 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2042,"mutex_wait_us":45,"thread_start_us":71,"threads_started":1}
I20260808 16:54:25.676857 29370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:25.669877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52440 (local address 127.8.24.1:40643)
0808 16:54:25.671533 (+ 1656us) server_negotiation.cc:206] Beginning negotiation
0808 16:54:25.671538 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 16:54:25.671940 (+ 402us) server_negotiation.cc:407] Connection header received
0808 16:54:25.671983 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:54:25.671986 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:54:25.672003 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:54:25.672072 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:54:25.672519 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:25.673644 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:25.675466 (+ 1822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:25.676217 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:25.676284 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:25.676336 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 16:54:25.676731 (+ 395us) server_negotiation.cc:299] Negotiation successful
0808 16:54:25.676779 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1617,"thread_start_us":45,"threads_started":1}
I20260808 16:54:25.676867 29373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:25.670076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39520 (local address 127.8.24.2:40639)
0808 16:54:25.670185 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0808 16:54:25.670189 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 16:54:25.672065 (+ 1876us) server_negotiation.cc:407] Connection header received
0808 16:54:25.672110 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:54:25.672113 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:54:25.672130 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:54:25.672204 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:54:25.672565 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:25.673682 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:25.675229 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:25.676021 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:25.676086 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:25.676143 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 16:54:25.676741 (+ 598us) server_negotiation.cc:299] Negotiation successful
0808 16:54:25.676786 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81,"thread_start_us":58,"threads_started":1}
I20260808 16:54:25.677542 28992 tablet_service.cc:1467] Processing CreateTablet for tablet ae8183e0179e4183b579b8fc6c19152c (DEFAULT_TABLE table=test-table [id=0a9b2a04a05049a9b86b3ebb5ad3279a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:54:25.677508 28770 tablet_service.cc:1467] Processing CreateTablet for tablet ae8183e0179e4183b579b8fc6c19152c (DEFAULT_TABLE table=test-table [id=0a9b2a04a05049a9b86b3ebb5ad3279a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:54:25.677866 28992 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae8183e0179e4183b579b8fc6c19152c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:54:25.677866 28770 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae8183e0179e4183b579b8fc6c19152c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:54:25.681861 29368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:25.669556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.3:41989 (local address 127.0.0.1:39982)
0808 16:54:25.671608 (+ 2052us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:25.671660 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 16:54:25.671730 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:54:25.672460 (+ 730us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:54:25.672460 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:54:25.672462 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:54:25.672493 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:54:25.672494 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:54:25.679323 (+ 6829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:54:25.679324 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:54:25.680390 (+ 1066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:54:25.680393 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:54:25.681636 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:54:25.681637 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:54:25.681699 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:25.681703 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 16:54:25.681762 (+ 59us) client_negotiation.cc:240] Negotiation successful
0808 16:54:25.681796 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2022,"thread_start_us":85,"threads_started":1}
I20260808 16:54:25.681900 29374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:25.670108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39982 (local address 127.8.24.3:41989)
0808 16:54:25.672176 (+ 2068us) server_negotiation.cc:206] Beginning negotiation
0808 16:54:25.672181 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 16:54:25.672190 (+ 9us) server_negotiation.cc:407] Connection header received
0808 16:54:25.672220 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:54:25.672221 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:54:25.672236 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:54:25.672302 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:54:25.677483 (+ 5181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:25.678298 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:25.680491 (+ 2193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:25.681540 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:25.681628 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:25.681684 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 16:54:25.681779 (+ 95us) server_negotiation.cc:299] Negotiation successful
0808 16:54:25.681823 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2025,"thread_start_us":62,"threads_started":1}
I20260808 16:54:25.682660 29377 tablet_bootstrap.cc:492] T ae8183e0179e4183b579b8fc6c19152c P cf6a0ebb4a2440bfa0f7595dbd8dad1e: Bootstrap starting.
I20260808 16:54:25.682705 29212 tablet_service.cc:1467] Processing CreateTablet for tablet ae8183e0179e4183b579b8fc6c19152c (DEFAULT_TABLE table=test-table [id=0a9b2a04a05049a9b86b3ebb5ad3279a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:54:25.683030 29212 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae8183e0179e4183b579b8fc6c19152c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:54:25.683506 29377 tablet_bootstrap.cc:654] T ae8183e0179e4183b579b8fc6c19152c P cf6a0ebb4a2440bfa0f7595dbd8dad1e: Neither blocks nor log segments found. Creating new log.
I20260808 16:54:25.683856 29377 log.cc:824] T ae8183e0179e4183b579b8fc6c19152c P cf6a0ebb4a2440bfa0f7595dbd8dad1e: Log is configured to *not* fsync() on all Append() calls
I20260808 16:54:25.684188 29376 tablet_bootstrap.cc:492] T ae8183e0179e4183b579b8fc6c19152c P abf681eae39e4d6bac1e70b80e4cd7a9: Bootstrap starting.
I20260808 16:54:25.684836 29376 tablet_bootstrap.cc:654] T ae8183e0179e4183b579b8fc6c19152c P abf681eae39e4d6bac1e70b80e4cd7a9: Neither blocks nor log segments found. Creating new log.
I20260808 16:54:25.685189 29376 log.cc:824] T ae8183e0179e4183b579b8fc6c19152c P abf681eae39e4d6bac1e70b80e4cd7a9: Log is configured to *not* fsync() on all Append() calls
I20260808 16:54:25.686426 29381 tablet_bootstrap.cc:492] T ae8183e0179e4183b579b8fc6c19152c P 1334561c8ce74535b255a792168ea9d4: Bootstrap starting.
I20260808 16:54:25.687057 29381 tablet_bootstrap.cc:654] T ae8183e0179e4183b579b8fc6c19152c P 1334561c8ce74535b255a792168ea9d4: Neither blocks nor log segments found. Creating new log.
I20260808 16:54:25.687377 29381 log.cc:824] T ae8183e0179e4183b579b8fc6c19152c P 1334561c8ce74535b255a792168ea9d4: Log is configured to *not* fsync() on all Append() calls
I20260808 16:54:25.687394 29377 tablet_bootstrap.cc:492] T ae8183e0179e4183b579b8fc6c19152c P cf6a0ebb4a2440bfa0f7595dbd8dad1e: No bootstrap required, opened a new log
I20260808 16:54:25.687458 29377 ts_tablet_manager.cc:1397] T ae8183e0179e4183b579b8fc6c19152c P cf6a0ebb4a2440bfa0f7595dbd8dad1e: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260808 16:54:25.689942 29377 raft_consensus.cc:348] T ae8183e0179e4183b579b8fc6c19152c P cf6a0ebb4a2440bfa0f7595dbd8dad1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1334561c8ce74535b255a792168ea9d4" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 41989 } } peers { permanent_uuid: "abf681eae39e4d6bac1e70b80e4cd7a9" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 40643 } } peers { permanent_uuid: "cf6a0ebb4a2440bfa0f7595dbd8dad1e" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 40639 } }
I20260808 16:54:25.690104 29377 raft_consensus.cc:374] T ae8183e0179e4183b579b8fc6c19152c P cf6a0ebb4a2440bfa0f7595dbd8dad1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:54:25.690143 29377 raft_consensus.cc:729] T ae8183e0179e4183b579b8fc6c19152c P cf6a0ebb4a2440bfa0f7595dbd8dad1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf6a0ebb4a2440bfa0f7595dbd8dad1e, State: Initialized, Role: FOLLOWER
I20260808 16:54:25.690274 29377 consensus_queue.cc:260] T ae8183e0179e4183b579b8fc6c19152c P cf6a0ebb4a2440bfa0f7595dbd8dad1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1334561c8ce74535b255a792168ea9d4" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 41989 } } peers { permanent_uuid: "abf681eae39e4d6bac1e70b80e4cd7a9" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 40643 } } peers { permanent_uuid: "cf6a0ebb4a2440bfa0f7595dbd8dad1e" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 40639 } }
I20260808 16:54:25.690546 29377 ts_tablet_manager.cc:1428] T ae8183e0179e4183b579b8fc6c19152c P cf6a0ebb4a2440bfa0f7595dbd8dad1e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 16:54:25.690675 29098 heartbeater.cc:503] Master 127.8.24.62:45771 was elected leader, sending a full tablet report...
I20260808 16:54:25.692305 29376 tablet_bootstrap.cc:492] T ae8183e0179e4183b579b8fc6c19152c P abf681eae39e4d6bac1e70b80e4cd7a9: No bootstrap required, opened a new log
I20260808 16:54:25.692413 29376 ts_tablet_manager.cc:1397] T ae8183e0179e4183b579b8fc6c19152c P abf681eae39e4d6bac1e70b80e4cd7a9: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20260808 16:54:25.692939 29381 tablet_bootstrap.cc:492] T ae8183e0179e4183b579b8fc6c19152c P 1334561c8ce74535b255a792168ea9d4: No bootstrap required, opened a new log
I20260808 16:54:25.693009 29381 ts_tablet_manager.cc:1397] T ae8183e0179e4183b579b8fc6c19152c P 1334561c8ce74535b255a792168ea9d4: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20260808 16:54:25.694610 29376 raft_consensus.cc:348] T ae8183e0179e4183b579b8fc6c19152c P abf681eae39e4d6bac1e70b80e4cd7a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1334561c8ce74535b255a792168ea9d4" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 41989 } } peers { permanent_uuid: "abf681eae39e4d6bac1e70b80e4cd7a9" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 40643 } } peers { permanent_uuid: "cf6a0ebb4a2440bfa0f7595dbd8dad1e" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 40639 } }
I20260808 16:54:25.694816 29376 raft_consensus.cc:374] T ae8183e0179e4183b579b8fc6c19152c P abf681eae39e4d6bac1e70b80e4cd7a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:54:25.694885 29376 raft_consensus.cc:729] T ae8183e0179e4183b579b8fc6c19152c P abf681eae39e4d6bac1e70b80e4cd7a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abf681eae39e4d6bac1e70b80e4cd7a9, State: Initialized, Role: FOLLOWER
I20260808 16:54:25.695045 29376 consensus_queue.cc:260] T ae8183e0179e4183b579b8fc6c19152c P abf681eae39e4d6bac1e70b80e4cd7a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1334561c8ce74535b255a792168ea9d4" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 41989 } } peers { permanent_uuid: "abf681eae39e4d6bac1e70b80e4cd7a9" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 40643 } } peers { permanent_uuid: "cf6a0ebb4a2440bfa0f7595dbd8dad1e" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 40639 } }
I20260808 16:54:25.695318 29376 ts_tablet_manager.cc:1428] T ae8183e0179e4183b579b8fc6c19152c P abf681eae39e4d6bac1e70b80e4cd7a9: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 16:54:25.695453 28885 heartbeater.cc:503] Master 127.8.24.62:45771 was elected leader, sending a full tablet report...
I20260808 16:54:25.695487 29381 raft_consensus.cc:348] T ae8183e0179e4183b579b8fc6c19152c P 1334561c8ce74535b255a792168ea9d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1334561c8ce74535b255a792168ea9d4" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 41989 } } peers { permanent_uuid: "abf681eae39e4d6bac1e70b80e4cd7a9" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 40643 } } peers { permanent_uuid: "cf6a0ebb4a2440bfa0f7595dbd8dad1e" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 40639 } }
I20260808 16:54:25.695643 29381 raft_consensus.cc:374] T ae8183e0179e4183b579b8fc6c19152c P 1334561c8ce74535b255a792168ea9d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:54:25.695675 29381 raft_consensus.cc:729] T ae8183e0179e4183b579b8fc6c19152c P 1334561c8ce74535b255a792168ea9d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1334561c8ce74535b255a792168ea9d4, State: Initialized, Role: FOLLOWER
I20260808 16:54:25.696038 29381 consensus_queue.cc:260] T ae8183e0179e4183b579b8fc6c19152c P 1334561c8ce74535b255a792168ea9d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1334561c8ce74535b255a792168ea9d4" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 41989 } } peers { permanent_uuid: "abf681eae39e4d6bac1e70b80e4cd7a9" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 40643 } } peers { permanent_uuid: "cf6a0ebb4a2440bfa0f7595dbd8dad1e" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 40639 } }
I20260808 16:54:25.696305 29381 ts_tablet_manager.cc:1428] T ae8183e0179e4183b579b8fc6c19152c P 1334561c8ce74535b255a792168ea9d4: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 16:54:25.696329 29325 heartbeater.cc:503] Master 127.8.24.62:45771 was elected leader, sending a full tablet report...
W20260808 16:54:25.698447 29103 tablet.cc:2369] T ae8183e0179e4183b579b8fc6c19152c P cf6a0ebb4a2440bfa0f7595dbd8dad1e: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 08 16:54:25 dist-test-slave-6zbq krb5kdc[26980](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208065, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for HTTP/127.8.24.20@KRBTEST.COM
I20260808 16:54:25.732208 29385 raft_consensus.cc:482] T ae8183e0179e4183b579b8fc6c19152c P 1334561c8ce74535b255a792168ea9d4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 16:54:25.732326 29385 raft_consensus.cc:504] T ae8183e0179e4183b579b8fc6c19152c P 1334561c8ce74535b255a792168ea9d4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1334561c8ce74535b255a792168ea9d4" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 41989 } } peers { permanent_uuid: "abf681eae39e4d6bac1e70b80e4cd7a9" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 40643 } } peers { permanent_uuid: "cf6a0ebb4a2440bfa0f7595dbd8dad1e" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 40639 } }
I20260808 16:54:25.732702 29385 leader_election.cc:288] T ae8183e0179e4183b579b8fc6c19152c P 1334561c8ce74535b255a792168ea9d4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers abf681eae39e4d6bac1e70b80e4cd7a9 (127.8.24.1:40643), cf6a0ebb4a2440bfa0f7595dbd8dad1e (127.8.24.2:40639)
I20260808 16:54:25.738511 29370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:25.733396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:44701 (local address 127.8.24.1:40643)
0808 16:54:25.733436 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 16:54:25.733438 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:54:25.733564 (+ 126us) server_negotiation.cc:407] Connection header received
0808 16:54:25.733673 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:54:25.733674 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:54:25.733686 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:54:25.733713 (+ 27us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:54:25.733937 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:25.734700 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:25.736610 (+ 1910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:25.737125 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:25.737168 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:25.737177 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 16:54:25.738406 (+ 1229us) server_negotiation.cc:299] Negotiation successful
0808 16:54:25.738448 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 16:54:25.738899 29388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:25.732926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:40643 (local address 127.8.24.3:44701)
0808 16:54:25.733505 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:25.733516 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 16:54:25.733590 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:54:25.733775 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:54:25.733776 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:54:25.733777 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:54:25.733862 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:54:25.733864 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:54:25.734912 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:54:25.734913 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:54:25.736145 (+ 1232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:54:25.736149 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:54:25.737246 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:54:25.737247 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:54:25.737310 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:25.737313 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 16:54:25.738770 (+ 1457us) client_negotiation.cc:240] Negotiation successful
0808 16:54:25.738811 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":561,"thread_start_us":98,"threads_started":1}
I20260808 16:54:25.739301 28779 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae8183e0179e4183b579b8fc6c19152c" candidate_uuid: "1334561c8ce74535b255a792168ea9d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abf681eae39e4d6bac1e70b80e4cd7a9" is_pre_election: true
I20260808 16:54:25.739466 28779 raft_consensus.cc:2454] T ae8183e0179e4183b579b8fc6c19152c P abf681eae39e4d6bac1e70b80e4cd7a9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1334561c8ce74535b255a792168ea9d4 in term 0.
I20260808 16:54:25.739703 29138 leader_election.cc:302] T ae8183e0179e4183b579b8fc6c19152c P 1334561c8ce74535b255a792168ea9d4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1334561c8ce74535b255a792168ea9d4, abf681eae39e4d6bac1e70b80e4cd7a9; no voters:
I20260808 16:54:25.739832 29385 raft_consensus.cc:2781] T ae8183e0179e4183b579b8fc6c19152c P 1334561c8ce74535b255a792168ea9d4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 16:54:25.739876 29385 raft_consensus.cc:482] T ae8183e0179e4183b579b8fc6c19152c P 1334561c8ce74535b255a792168ea9d4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 16:54:25.739900 29385 raft_consensus.cc:3037] T ae8183e0179e4183b579b8fc6c19152c P 1334561c8ce74535b255a792168ea9d4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:54:25.740760 29385 raft_consensus.cc:504] T ae8183e0179e4183b579b8fc6c19152c P 1334561c8ce74535b255a792168ea9d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1334561c8ce74535b255a792168ea9d4" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 41989 } } peers { permanent_uuid: "abf681eae39e4d6bac1e70b80e4cd7a9" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 40643 } } peers { permanent_uuid: "cf6a0ebb4a2440bfa0f7595dbd8dad1e" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 40639 } }
I20260808 16:54:25.740880 29385 leader_election.cc:288] T ae8183e0179e4183b579b8fc6c19152c P 1334561c8ce74535b255a792168ea9d4 [CANDIDATE]: Term 1 election: Requested vote from peers abf681eae39e4d6bac1e70b80e4cd7a9 (127.8.24.1:40643), cf6a0ebb4a2440bfa0f7595dbd8dad1e (127.8.24.2:40639)
I20260808 16:54:25.741041 28779 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae8183e0179e4183b579b8fc6c19152c" candidate_uuid: "1334561c8ce74535b255a792168ea9d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abf681eae39e4d6bac1e70b80e4cd7a9"
I20260808 16:54:25.741165 28779 raft_consensus.cc:3037] T ae8183e0179e4183b579b8fc6c19152c P abf681eae39e4d6bac1e70b80e4cd7a9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:54:25.741420 29373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:25.733404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:42761 (local address 127.8.24.2:40639)
0808 16:54:25.733450 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 16:54:25.733452 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:54:25.736307 (+ 2855us) server_negotiation.cc:407] Connection header received
0808 16:54:25.736397 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:54:25.736398 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:54:25.736410 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:54:25.736437 (+ 27us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:54:25.737386 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:25.738045 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:25.740404 (+ 2359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:25.740918 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:25.740959 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:25.740969 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 16:54:25.741319 (+ 350us) server_negotiation.cc:299] Negotiation successful
0808 16:54:25.741362 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 16:54:25.741744 29386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:25.733117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.2:40639 (local address 127.8.24.3:42761)
0808 16:54:25.736257 (+ 3140us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:25.736266 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 16:54:25.736331 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:54:25.736491 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:54:25.736492 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:54:25.736493 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:54:25.736543 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:54:25.736544 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:54:25.738134 (+ 1590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:54:25.738135 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:54:25.740312 (+ 2177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:54:25.740315 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:54:25.741082 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:54:25.741082 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:54:25.741209 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:25.741213 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 16:54:25.741634 (+ 421us) client_negotiation.cc:240] Negotiation successful
0808 16:54:25.741672 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3123,"thread_start_us":67,"threads_started":1}
I20260808 16:54:25.741981 28779 raft_consensus.cc:2454] T ae8183e0179e4183b579b8fc6c19152c P abf681eae39e4d6bac1e70b80e4cd7a9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1334561c8ce74535b255a792168ea9d4 in term 1.
I20260808 16:54:25.742161 29138 leader_election.cc:302] T ae8183e0179e4183b579b8fc6c19152c P 1334561c8ce74535b255a792168ea9d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1334561c8ce74535b255a792168ea9d4, abf681eae39e4d6bac1e70b80e4cd7a9; no voters:
I20260808 16:54:25.742128 29032 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae8183e0179e4183b579b8fc6c19152c" candidate_uuid: "1334561c8ce74535b255a792168ea9d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf6a0ebb4a2440bfa0f7595dbd8dad1e"
I20260808 16:54:25.742235 29385 raft_consensus.cc:2781] T ae8183e0179e4183b579b8fc6c19152c P 1334561c8ce74535b255a792168ea9d4 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:54:25.742236 29032 raft_consensus.cc:3037] T ae8183e0179e4183b579b8fc6c19152c P cf6a0ebb4a2440bfa0f7595dbd8dad1e [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:54:25.742377 29385 raft_consensus.cc:686] T ae8183e0179e4183b579b8fc6c19152c P 1334561c8ce74535b255a792168ea9d4 [term 1 LEADER]: Becoming Leader. State: Replica: 1334561c8ce74535b255a792168ea9d4, State: Running, Role: LEADER
I20260808 16:54:25.742458 29385 consensus_queue.cc:237] T ae8183e0179e4183b579b8fc6c19152c P 1334561c8ce74535b255a792168ea9d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1334561c8ce74535b255a792168ea9d4" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 41989 } } peers { permanent_uuid: "abf681eae39e4d6bac1e70b80e4cd7a9" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 40643 } } peers { permanent_uuid: "cf6a0ebb4a2440bfa0f7595dbd8dad1e" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 40639 } }
I20260808 16:54:25.743052 29032 raft_consensus.cc:2454] T ae8183e0179e4183b579b8fc6c19152c P cf6a0ebb4a2440bfa0f7595dbd8dad1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1334561c8ce74535b255a792168ea9d4 in term 1.
I20260808 16:54:25.743202 28563 catalog_manager.cc:5495] T ae8183e0179e4183b579b8fc6c19152c P 1334561c8ce74535b255a792168ea9d4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1334561c8ce74535b255a792168ea9d4 (127.8.24.3). New cstate: current_term: 1 leader_uuid: "1334561c8ce74535b255a792168ea9d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1334561c8ce74535b255a792168ea9d4" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 41989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "abf681eae39e4d6bac1e70b80e4cd7a9" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 40643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf6a0ebb4a2440bfa0f7595dbd8dad1e" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 40639 } health_report { overall_health: UNKNOWN } } }
I20260808 16:54:25.742117 28995 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae8183e0179e4183b579b8fc6c19152c" candidate_uuid: "1334561c8ce74535b255a792168ea9d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf6a0ebb4a2440bfa0f7595dbd8dad1e" is_pre_election: true
I20260808 16:54:25.743664 28995 raft_consensus.cc:2362] T ae8183e0179e4183b579b8fc6c19152c P cf6a0ebb4a2440bfa0f7595dbd8dad1e [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 1334561c8ce74535b255a792168ea9d4 in term 1. Re-sending same reply.
W20260808 16:54:25.756382 28888 tablet.cc:2369] T ae8183e0179e4183b579b8fc6c19152c P abf681eae39e4d6bac1e70b80e4cd7a9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 16:54:25.767663 29374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:25.763977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46754 (local address 127.8.24.3:41989)
0808 16:54:25.764042 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 16:54:25.764044 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:54:25.764051 (+ 7us) server_negotiation.cc:407] Connection header received
0808 16:54:25.764082 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:54:25.764083 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:54:25.764101 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:54:25.764166 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 16:54:25.764366 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:25.765082 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:25.766321 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:25.766669 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:25.766733 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:25.767135 (+ 402us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 16:54:25.767258 (+ 123us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 16:54:25.767337 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 16:54:25.767562 (+ 225us) server_negotiation.cc:299] Negotiation successful
0808 16:54:25.767582 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 16:54:25.769562 28995 raft_consensus.cc:1261] T ae8183e0179e4183b579b8fc6c19152c P cf6a0ebb4a2440bfa0f7595dbd8dad1e [term 1 FOLLOWER]: Refusing update from remote peer 1334561c8ce74535b255a792168ea9d4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:54:25.769898 29385 consensus_queue.cc:1035] T ae8183e0179e4183b579b8fc6c19152c P 1334561c8ce74535b255a792168ea9d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf6a0ebb4a2440bfa0f7595dbd8dad1e" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 40639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:54:25.770416 28779 raft_consensus.cc:1261] T ae8183e0179e4183b579b8fc6c19152c P abf681eae39e4d6bac1e70b80e4cd7a9 [term 1 FOLLOWER]: Refusing update from remote peer 1334561c8ce74535b255a792168ea9d4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:54:25.770691 29389 consensus_queue.cc:1035] T ae8183e0179e4183b579b8fc6c19152c P 1334561c8ce74535b255a792168ea9d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "abf681eae39e4d6bac1e70b80e4cd7a9" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 40643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:54:25.778673 29398 mvcc.cc:205] Tried to move back new op lower bound from 7316308237389381632 to 7316308237281566720. Current Snapshot: MvccSnapshot[applied={T|T < 7316308237389381632}]
I20260808 16:54:25.780867 29396 mvcc.cc:205] Tried to move back new op lower bound from 7316308237389381632 to 7316308237281566720. Current Snapshot: MvccSnapshot[applied={T|T < 7316308237389381632}]
I20260808 16:54:25.783607 29394 mvcc.cc:205] Tried to move back new op lower bound from 7316308237389381632 to 7316308237281566720. Current Snapshot: MvccSnapshot[applied={T|T < 7316308237389381632}]
I20260808 16:54:25.788321 28563 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35410:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 16:54:25.788424 28563 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35410:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 16:54:25.789150 28563 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f05881afb75648c89dedd2f5d476e96a: Sending DeleteTablet for 3 replicas of tablet ae8183e0179e4183b579b8fc6c19152c
I20260808 16:54:25.789716 29212 tablet_service.cc:1514] Processing DeleteTablet for tablet ae8183e0179e4183b579b8fc6c19152c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:54:25 UTC) from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:39982
I20260808 16:54:25.789937 28770 tablet_service.cc:1514] Processing DeleteTablet for tablet ae8183e0179e4183b579b8fc6c19152c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:54:25 UTC) from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:52440
I20260808 16:54:25.790159 28992 tablet_service.cc:1514] Processing DeleteTablet for tablet ae8183e0179e4183b579b8fc6c19152c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:54:25 UTC) from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:39520
I20260808 16:54:25.790901 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 28629
I20260808 16:54:25.798451 29409 tablet_replica.cc:330] T ae8183e0179e4183b579b8fc6c19152c P 1334561c8ce74535b255a792168ea9d4: stopping tablet replica
I20260808 16:54:25.798568 29409 raft_consensus.cc:2229] T ae8183e0179e4183b579b8fc6c19152c P 1334561c8ce74535b255a792168ea9d4 [term 1 LEADER]: Raft consensus shutting down.
I20260808 16:54:25.798704 29409 raft_consensus.cc:2258] T ae8183e0179e4183b579b8fc6c19152c P 1334561c8ce74535b255a792168ea9d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:54:25.799706 29412 tablet_replica.cc:330] T ae8183e0179e4183b579b8fc6c19152c P cf6a0ebb4a2440bfa0f7595dbd8dad1e: stopping tablet replica
I20260808 16:54:25.799816 29412 raft_consensus.cc:2229] T ae8183e0179e4183b579b8fc6c19152c P cf6a0ebb4a2440bfa0f7595dbd8dad1e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 16:54:25.799922 29412 raft_consensus.cc:2258] T ae8183e0179e4183b579b8fc6c19152c P cf6a0ebb4a2440bfa0f7595dbd8dad1e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:54:25.801646 29412 ts_tablet_manager.cc:1905] T ae8183e0179e4183b579b8fc6c19152c P cf6a0ebb4a2440bfa0f7595dbd8dad1e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:54:25.802970 29409 ts_tablet_manager.cc:1905] T ae8183e0179e4183b579b8fc6c19152c P 1334561c8ce74535b255a792168ea9d4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:54:25.804666 29409 ts_tablet_manager.cc:1918] T ae8183e0179e4183b579b8fc6c19152c P 1334561c8ce74535b255a792168ea9d4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 16:54:25.804742 29409 log.cc:1196] T ae8183e0179e4183b579b8fc6c19152c P 1334561c8ce74535b255a792168ea9d4: Deleting WAL directory at /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ts-2/wal/wals/ae8183e0179e4183b579b8fc6c19152c
I20260808 16:54:25.805012 29409 ts_tablet_manager.cc:1939] T ae8183e0179e4183b579b8fc6c19152c P 1334561c8ce74535b255a792168ea9d4: Deleting consensus metadata
I20260808 16:54:25.805405 28546 catalog_manager.cc:4841] TS 1334561c8ce74535b255a792168ea9d4 (127.8.24.3:41989): tablet ae8183e0179e4183b579b8fc6c19152c (table test-table [id=0a9b2a04a05049a9b86b3ebb5ad3279a]) successfully deleted
I20260808 16:54:25.807018 29412 ts_tablet_manager.cc:1918] T ae8183e0179e4183b579b8fc6c19152c P cf6a0ebb4a2440bfa0f7595dbd8dad1e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 16:54:25.807093 29412 log.cc:1196] T ae8183e0179e4183b579b8fc6c19152c P cf6a0ebb4a2440bfa0f7595dbd8dad1e: Deleting WAL directory at /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786207919384338-8288-0/minicluster-data/ts-1/wal/wals/ae8183e0179e4183b579b8fc6c19152c
I20260808 16:54:25.807358 29412 ts_tablet_manager.cc:1939] T ae8183e0179e4183b579b8fc6c19152c P cf6a0ebb4a2440bfa0f7595dbd8dad1e: Deleting consensus metadata
I20260808 16:54:25.807731 28546 catalog_manager.cc:4841] TS cf6a0ebb4a2440bfa0f7595dbd8dad1e (127.8.24.2:40639): tablet ae8183e0179e4183b579b8fc6c19152c (table test-table [id=0a9b2a04a05049a9b86b3ebb5ad3279a]) successfully deleted
W20260808 16:54:25.812045 28546 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.24.1:40643 (error 108)
I20260808 16:54:25.812731 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 28892
I20260808 16:54:25.834861 29368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:25.834722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:40643 (local address 127.0.0.1:57208)
0808 16:54:25.834781 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:25.834821 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.1:40643: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260808 16:54:25.835001 28546 catalog_manager.cc:4584] TS abf681eae39e4d6bac1e70b80e4cd7a9 (127.8.24.1:40643): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ae8183e0179e4183b579b8fc6c19152c: Network error: Client connection negotiation failed: client connection to 127.8.24.1:40643: connect: Connection refused (error 111)
I20260808 16:54:25.877043 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 29106
I20260808 16:54:25.888146 29368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:25.888024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:40643 (local address 127.0.0.1:57218)
0808 16:54:25.888076 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:25.888101 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.1:40643: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260808 16:54:25.888377 29368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:25.888330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:40643 (local address 127.0.0.1:57228)
0808 16:54:25.888350 (+ 20us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:25.888365 (+ 15us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.1:40643: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":13}
I20260808 16:54:25.911032 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 28506
I20260808 16:54:25.922170 8288 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260808 16:54:26.265683 8288 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260808 16:54:26.265772 8288 mini_ranger.cc:66] Stopping Ranger...
I20260808 16:54:26.609989 8288 mini_ranger.cc:68] Stopped Ranger
2026-08-08 16:54:26.610 UTC [27027] LOG: received smart shutdown request
2026-08-08 16:54:26.614 UTC [27027] LOG: background worker "logical replication launcher" (PID 27039) exited with exit code 1
2026-08-08 16:54:26.615 UTC [27032] LOG: shutting down
2026-08-08 16:54:26.739 UTC [27027] LOG: database system is shut down
2026-08-08T16:54:26Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (45588 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786207919384338-8288-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 16:54:26 dist-test-slave-6zbq krb5kdc[29425](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 16:54:26 dist-test-slave-6zbq krb5kdc[29425](info): set up 2 sockets
Aug 08 16:54:26 dist-test-slave-6zbq krb5kdc[29425](info): commencing operation
krb5kdc: starting...
W20260808 16:54:27.937592 8288 mini_kdc.cc:121] Time spent starting KDC: real 1.098s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786207919384338-8288-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786207919384338-8288-0/krb5kdc/test-user.keytab.
Aug 08 16:54:28 dist-test-slave-6zbq krb5kdc[29425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208068, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T16:54:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T16:54:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.24.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
I20260808 16:54:28.042361 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786207919384338-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786207919384338-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786207919384338-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:45743
--webserver_interface=127.8.24.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:45551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:45743,127.8.24.61:33015,127.8.24.60:46443
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:54:28.119190 29449 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:54:28.119359 29449 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:54:28.119378 29449 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:54:28.120450 29449 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 16:54:28.120488 29449 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:54:28.120501 29449 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:54:28.120525 29449 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 16:54:28.120548 29449 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 16:54:28.121990 29449 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:45551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786207919384338-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786207919384338-8288-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:45743,127.8.24.61:33015,127.8.24.60:46443
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:45743
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.24.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29449
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786207919384338-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:54:28.122195 29449 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:54:28.122409 29449 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:54:28.126595 29457 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:54:28.126667 29449 server_base.cc:1034] running on GCE node
W20260808 16:54:28.126600 29455 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:54:28.126583 29459 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:54:28.126993 29449 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:54:28.127249 29449 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:54:28.128381 29449 hybrid_clock.cc:648] HybridClock initialized: now 1786208068128367 us; error 30 us; skew 500 ppm
Aug 08 16:54:28 dist-test-slave-6zbq krb5kdc[29425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208068, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:54:28.133117 29449 init.cc:377] Logged in from keytab as kudu/127.8.24.62@KRBTEST.COM (short username kudu)
I20260808 16:54:28.133523 29449 webserver.cc:460] Webserver started at http://127.8.24.62:40523/ using document root <none> and password file <none>
I20260808 16:54:28.133701 29449 fs_manager.cc:359] Metadata directory not provided
I20260808 16:54:28.133744 29449 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:54:28.133854 29449 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:54:28.134807 29449 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786207919384338-8288-0/minicluster-data/master-0/data/instance:
uuid: "2a79a0a7cfd94418be2c577f70bb56c6"
format_stamp: "Formatted at 2026-08-08 16:54:28 on dist-test-slave-6zbq"
server_key: "fec99e91ba88b951c297e3122f9b56dd"
server_key_iv: "edabe9dd1bc734c299dc32699d046256"
server_key_version: "encryptionkey@0"
I20260808 16:54:28.135102 29449 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786207919384338-8288-0/minicluster-data/master-0/wal/instance:
uuid: "2a79a0a7cfd94418be2c577f70bb56c6"
format_stamp: "Formatted at 2026-08-08 16:54:28 on dist-test-slave-6zbq"
server_key: "fec99e91ba88b951c297e3122f9b56dd"
server_key_iv: "edabe9dd1bc734c299dc32699d046256"
server_key_version: "encryptionkey@0"
I20260808 16:54:28.136229 29449 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 16:54:28.136819 29474 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:54:28.136966 29449 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 16:54:28.137045 29449 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786207919384338-8288-0/minicluster-data/master-0/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786207919384338-8288-0/minicluster-data/master-0/wal
uuid: "2a79a0a7cfd94418be2c577f70bb56c6"
format_stamp: "Formatted at 2026-08-08 16:54:28 on dist-test-slave-6zbq"
server_key: "fec99e91ba88b951c297e3122f9b56dd"
server_key_iv: "edabe9dd1bc734c299dc32699d046256"
server_key_version: "encryptionkey@0"
I20260808 16:54:28.137101 29449 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786207919384338-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786207919384338-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786207919384338-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:54:28.149184 29449 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:54:28.150751 29449 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:54:28.150870 29449 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:54:28.154515 29449 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.62:45743
I20260808 16:54:28.154551 29563 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.62:45743 every 8 connection(s)
I20260808 16:54:28.154829 29449 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
I20260808 16:54:28.155558 29564 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:54:28.156019 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 29449
I20260808 16:54:28.156111 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786207919384338-8288-0/minicluster-data/master-0/wal/instance
I20260808 16:54:28.156286 8288 external_mini_cluster.cc:1428] Setting key d4e3b4bb90a2937be8bdc93805b17cf7
I20260808 16:54:28.157002 29564 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.24.62" port: 45743 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 16:54:28 dist-test-slave-6zbq krb5kdc[29425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208068, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.24.61@KRBTEST.COM; defaulting to no policy
I20260808 16:54:28.166270 29575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:28.157638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:45743 (local address 127.0.0.1:41318)
0808 16:54:28.157849 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:28.157882 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 16:54:28.158295 (+ 413us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:54:28.158565 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:54:28.158566 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:54:28.158779 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:54:28.158919 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:54:28.158923 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:54:28.160343 (+ 1420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:54:28.160344 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:54:28.160980 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:54:28.160981 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:54:28.161630 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:54:28.161630 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:54:28.161682 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:28.162094 (+ 412us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:54:28.162101 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:54:28.163545 (+ 1444us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:54:28.165080 (+ 1535us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:54:28.165086 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:54:28.165088 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:54:28.165301 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:54:28.165498 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:54:28.165498 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:54:28.165499 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:54:28.165566 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:54:28.165837 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:54:28.165840 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:54:28.165886 (+ 46us) client_negotiation.cc:769] Sending connection context
0808 16:54:28.165946 (+ 60us) client_negotiation.cc:240] Negotiation successful
0808 16:54:28.166089 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":90,"threads_started":1}
I20260808 16:54:28.166272 29577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:28.157640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41318 (local address 127.8.24.62:45743)
0808 16:54:28.157836 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0808 16:54:28.157844 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 16:54:28.157960 (+ 116us) server_negotiation.cc:407] Connection header received
0808 16:54:28.158443 (+ 483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:54:28.158446 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:54:28.158467 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:54:28.158567 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:54:28.159007 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:28.160208 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:28.161057 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:28.161550 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:28.161659 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:28.163630 (+ 1971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:54:28.163639 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:54:28.163639 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:54:28.163646 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:54:28.165008 (+ 1362us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:54:28.165370 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:54:28.165373 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:54:28.165373 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:54:28.165448 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:54:28.165606 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:54:28.165607 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:54:28.165607 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:54:28.165776 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:54:28.165850 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 16:54:28.165987 (+ 137us) server_negotiation.cc:299] Negotiation successful
0808 16:54:28.166131 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":73,"server-negotiator.queue_time_us":153,"thread_start_us":91,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.61@KRBTEST.COM" created.
I20260808 16:54:28.166916 29564 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.24.61" port: 33015 } has no permanent_uuid. Determining permanent_uuid...
I20260808 16:54:28.167199 29575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:28.167101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.61:33015 (local address 127.0.0.1:40530)
0808 16:54:28.167137 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:28.167177 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.61:33015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260808 16:54:28.167310 29481 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.24.61:33015: connect: Connection refused (error 111)
I20260808 16:54:28.167665 29575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:28.167581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.61:33015 (local address 127.0.0.1:40546)
0808 16:54:28.167621 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:28.167644 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.61:33015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260808 16:54:28.167762 29564 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.24.61:33015: Network error: Client connection negotiation failed: client connection to 127.8.24.61:33015: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.61.keytab.
Entry for principal kudu/127.8.24.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.61.keytab.
I20260808 16:54:28.178368 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786207919384338-8288-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786207919384338-8288-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786207919384338-8288-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786207919384338-8288-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.61:33015
--webserver_interface=127.8.24.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.61.keytab
--principal=kudu/127.8.24.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:45551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:45743,127.8.24.61:33015,127.8.24.60:46443
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 16:54:28.216950 29564 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.24.61" port: 33015 } attempt: 1
I20260808 16:54:28.217296 29575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:28.217188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.61:33015 (local address 127.0.0.1:40548)
0808 16:54:28.217241 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:28.217273 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.61:33015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260808 16:54:28.217648 29575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:28.217553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.61:33015 (local address 127.0.0.1:40562)
0808 16:54:28.217598 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:28.217626 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.61:33015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260808 16:54:28.217739 29564 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.24.61:33015: Network error: Client connection negotiation failed: client connection to 127.8.24.61:33015: connect: Connection refused (error 111)
W20260808 16:54:28.255136 29582 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:54:28.255292 29582 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:54:28.255312 29582 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:54:28.256325 29582 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 16:54:28.256356 29582 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:54:28.256367 29582 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:54:28.256378 29582 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 16:54:28.256393 29582 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 16:54:28.257772 29582 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:45551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786207919384338-8288-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786207919384338-8288-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:45743,127.8.24.61:33015,127.8.24.60:46443
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.61.keytab
--principal=kudu/127.8.24.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.61:33015
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786207919384338-8288-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.8.24.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29582
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786207919384338-8288-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:54:28.257936 29582 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:54:28.258091 29582 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:54:28.262238 29590 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:54:28.262238 29592 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:54:28.262411 29582 server_base.cc:1034] running on GCE node
W20260808 16:54:28.262248 29589 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:54:28.262718 29582 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:54:28.262967 29582 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:54:28.264093 29582 hybrid_clock.cc:648] HybridClock initialized: now 1786208068264084 us; error 24 us; skew 500 ppm
Aug 08 16:54:28 dist-test-slave-6zbq krb5kdc[29425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208068, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:54:28.266214 29582 init.cc:377] Logged in from keytab as kudu/127.8.24.61@KRBTEST.COM (short username kudu)
I20260808 16:54:28.266606 29582 webserver.cc:460] Webserver started at http://127.8.24.61:40923/ using document root <none> and password file <none>
I20260808 16:54:28.266785 29582 fs_manager.cc:359] Metadata directory not provided
I20260808 16:54:28.266821 29582 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:54:28.266904 29582 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:54:28.267778 29582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786207919384338-8288-0/minicluster-data/master-1/data/instance:
uuid: "93ead526500149af99a55fe16fba1e75"
format_stamp: "Formatted at 2026-08-08 16:54:28 on dist-test-slave-6zbq"
server_key: "5896a4d2b71e0ee41d63aa04afb54535"
server_key_iv: "eeec8f0dfbf4594a5d4f11dd3f7cf2b7"
server_key_version: "encryptionkey@0"
I20260808 16:54:28.268036 29582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786207919384338-8288-0/minicluster-data/master-1/wal/instance:
uuid: "93ead526500149af99a55fe16fba1e75"
format_stamp: "Formatted at 2026-08-08 16:54:28 on dist-test-slave-6zbq"
server_key: "5896a4d2b71e0ee41d63aa04afb54535"
server_key_iv: "eeec8f0dfbf4594a5d4f11dd3f7cf2b7"
server_key_version: "encryptionkey@0"
I20260808 16:54:28.269109 29582 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 16:54:28.269678 29600 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:54:28.269821 29582 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 16:54:28.269899 29582 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786207919384338-8288-0/minicluster-data/master-1/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786207919384338-8288-0/minicluster-data/master-1/wal
uuid: "93ead526500149af99a55fe16fba1e75"
format_stamp: "Formatted at 2026-08-08 16:54:28 on dist-test-slave-6zbq"
server_key: "5896a4d2b71e0ee41d63aa04afb54535"
server_key_iv: "eeec8f0dfbf4594a5d4f11dd3f7cf2b7"
server_key_version: "encryptionkey@0"
I20260808 16:54:28.269956 29582 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786207919384338-8288-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786207919384338-8288-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786207919384338-8288-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:54:28.283643 29582 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:54:28.285118 29582 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:54:28.285241 29582 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:54:28.285869 29564 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.24.61" port: 33015 } attempt: 2
I20260808 16:54:28.286183 29575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:28.286051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.61:33015 (local address 127.0.0.1:40572)
0808 16:54:28.286115 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:28.286152 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.61:33015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260808 16:54:28.286528 29575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:28.286445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.61:33015 (local address 127.0.0.1:40574)
0808 16:54:28.286484 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:28.286508 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.61:33015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260808 16:54:28.286620 29564 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.24.61:33015: Network error: Client connection negotiation failed: client connection to 127.8.24.61:33015: connect: Connection refused (error 111)
I20260808 16:54:28.288851 29582 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.61:33015
I20260808 16:54:28.288882 29682 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.61:33015 every 8 connection(s)
I20260808 16:54:28.289150 29582 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786207919384338-8288-0/minicluster-data/master-1/data/info.pb
I20260808 16:54:28.289871 29683 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:54:28.291205 29683 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.24.62" port: 45743 } has no permanent_uuid. Determining permanent_uuid...
I20260808 16:54:28.292097 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 29582
I20260808 16:54:28.292200 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786207919384338-8288-0/minicluster-data/master-1/wal/instance
I20260808 16:54:28.292402 8288 external_mini_cluster.cc:1428] Setting key 72bc8ef89d3424ce3749802e859f6f1f
Aug 08 16:54:28 dist-test-slave-6zbq krb5kdc[29425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208068, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.61@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260808 16:54:28.299028 29577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:28.291752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41334 (local address 127.8.24.62:45743)
0808 16:54:28.291798 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 16:54:28.291800 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:54:28.291982 (+ 182us) server_negotiation.cc:407] Connection header received
0808 16:54:28.292519 (+ 537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:54:28.292519 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:54:28.292531 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:54:28.292570 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:54:28.293249 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:28.293729 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:28.294692 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:28.294919 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:28.294975 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:28.296669 (+ 1694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:54:28.296670 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:54:28.296671 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:54:28.296685 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:54:28.297863 (+ 1178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:54:28.298199 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:54:28.298200 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:54:28.298200 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:54:28.298252 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:54:28.298485 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:54:28.298486 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:54:28.298486 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:54:28.298586 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:54:28.298642 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 16:54:28.298872 (+ 230us) server_negotiation.cc:299] Negotiation successful
0808 16:54:28.298952 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 16:54:28.299162 29685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:28.291741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:45743 (local address 127.0.0.1:41334)
0808 16:54:28.291882 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:28.291911 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0808 16:54:28.292433 (+ 522us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:54:28.292632 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:54:28.292636 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:54:28.292916 (+ 280us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:54:28.293163 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:54:28.293169 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:54:28.293821 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:54:28.293822 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:54:28.294620 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:54:28.294623 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:54:28.294979 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:54:28.294980 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:54:28.295070 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:28.295446 (+ 376us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:54:28.295453 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:54:28.296588 (+ 1135us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:54:28.297933 (+ 1345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:54:28.297939 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:54:28.297941 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:54:28.298134 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:54:28.298304 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:54:28.298304 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:54:28.298305 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:54:28.298430 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:54:28.298639 (+ 209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:54:28.298645 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:54:28.298765 (+ 120us) client_negotiation.cc:769] Sending connection context
0808 16:54:28.298845 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 16:54:28.298978 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":74,"threads_started":1}
WARNING: no policy specified for kudu/127.8.24.60@KRBTEST.COM; defaulting to no policy
I20260808 16:54:28.299629 29683 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.24.61" port: 33015 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.60@KRBTEST.COM" created.
Aug 08 16:54:28 dist-test-slave-6zbq krb5kdc[29425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208068, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.61@KRBTEST.COM for kudu/127.8.24.61@KRBTEST.COM
I20260808 16:54:28.305923 29685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:28.299812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.61:33015 (local address 127.0.0.1:40588)
0808 16:54:28.299847 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:28.299856 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 16:54:28.299892 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:54:28.300155 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:54:28.300156 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:54:28.300261 (+ 105us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:54:28.300302 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:54:28.300303 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:54:28.301142 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:54:28.301143 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:54:28.301655 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:54:28.301656 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:54:28.302082 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:54:28.302083 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:54:28.302152 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:28.302517 (+ 365us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:54:28.302522 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:54:28.303590 (+ 1068us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:54:28.305021 (+ 1431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:54:28.305021 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:54:28.305022 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:54:28.305170 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:54:28.305338 (+ 168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:54:28.305339 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:54:28.305339 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:54:28.305389 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:54:28.305610 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:54:28.305611 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:54:28.305645 (+ 34us) client_negotiation.cc:769] Sending connection context
0808 16:54:28.305712 (+ 67us) client_negotiation.cc:240] Negotiation successful
0808 16:54:28.305801 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21}
I20260808 16:54:28.305991 29690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:28.299874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40588 (local address 127.8.24.61:33015)
0808 16:54:28.300019 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0808 16:54:28.300023 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 16:54:28.300040 (+ 17us) server_negotiation.cc:407] Connection header received
0808 16:54:28.300057 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:54:28.300059 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:54:28.300069 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:54:28.300144 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:54:28.300354 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:28.301056 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:28.301711 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:28.302023 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:28.302076 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:28.303681 (+ 1605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:54:28.303687 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:54:28.303687 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:54:28.303693 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:54:28.304946 (+ 1253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:54:28.305236 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:54:28.305238 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:54:28.305239 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:54:28.305286 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:54:28.305438 (+ 152us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:54:28.305438 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:54:28.305439 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:54:28.305553 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:54:28.305617 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 16:54:28.305746 (+ 129us) server_negotiation.cc:299] Negotiation successful
0808 16:54:28.305851 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":45,"server-negotiator.queue_time_us":119,"thread_start_us":83,"threads_started":1}
I20260808 16:54:28.306444 29683 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.24.60" port: 46443 } has no permanent_uuid. Determining permanent_uuid...
I20260808 16:54:28.306733 29685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:28.306631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.60:46443 (local address 127.0.0.1:37688)
0808 16:54:28.306668 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:28.306714 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.60:46443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260808 16:54:28.306864 29602 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.24.60:46443: connect: Connection refused (error 111)
I20260808 16:54:28.307231 29685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:28.307172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.60:46443 (local address 127.0.0.1:37694)
0808 16:54:28.307197 (+ 25us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:28.307217 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.60:46443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
W20260808 16:54:28.307327 29683 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.24.60:46443: Network error: Client connection negotiation failed: client connection to 127.8.24.60:46443: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.60.keytab.
Entry for principal kudu/127.8.24.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.60.keytab.
I20260808 16:54:28.315227 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786207919384338-8288-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786207919384338-8288-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786207919384338-8288-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786207919384338-8288-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.60:46443
--webserver_interface=127.8.24.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.60.keytab
--principal=kudu/127.8.24.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:45551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:45743,127.8.24.61:33015,127.8.24.60:46443
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 16:54:28.356528 29683 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.24.60" port: 46443 } attempt: 1
I20260808 16:54:28.356889 29685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:28.356795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.60:46443 (local address 127.0.0.1:37702)
0808 16:54:28.356843 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:28.356866 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.60:46443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260808 16:54:28.357192 29685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:28.357113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.60:46443 (local address 127.0.0.1:37708)
0808 16:54:28.357150 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:28.357171 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.60:46443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260808 16:54:28.357270 29683 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.24.60:46443: Network error: Client connection negotiation failed: client connection to 127.8.24.60:46443: connect: Connection refused (error 111)
I20260808 16:54:28.377771 29564 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.24.61" port: 33015 } attempt: 3
Aug 08 16:54:28 dist-test-slave-6zbq krb5kdc[29425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208068, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for kudu/127.8.24.61@KRBTEST.COM
I20260808 16:54:28.384408 29690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:28.378129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40604 (local address 127.8.24.61:33015)
0808 16:54:28.378191 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 16:54:28.378193 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:54:28.378227 (+ 34us) server_negotiation.cc:407] Connection header received
0808 16:54:28.378253 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:54:28.378254 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:54:28.378264 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:54:28.378309 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:54:28.378632 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:28.379379 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:28.380053 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:28.380350 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:28.380441 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:28.382197 (+ 1756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:54:28.382198 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:54:28.382198 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:54:28.382205 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:54:28.383420 (+ 1215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:54:28.383798 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:54:28.383799 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:54:28.383799 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:54:28.383842 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:54:28.384011 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:54:28.384011 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:54:28.384011 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:54:28.384099 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:54:28.384143 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0808 16:54:28.384290 (+ 147us) server_negotiation.cc:299] Negotiation successful
0808 16:54:28.384349 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 16:54:28.384467 29575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:28.378077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.61:33015 (local address 127.0.0.1:40604)
0808 16:54:28.378117 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:28.378128 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 16:54:28.378189 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:54:28.378325 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:54:28.378326 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:54:28.378488 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:54:28.378553 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:54:28.378554 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:54:28.379484 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:54:28.379485 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:54:28.379969 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:54:28.379970 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:54:28.380444 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:54:28.380444 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:54:28.380496 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:28.380888 (+ 392us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:54:28.380893 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:54:28.382113 (+ 1220us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:54:28.383502 (+ 1389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:54:28.383503 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:54:28.383503 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:54:28.383730 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:54:28.383893 (+ 163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:54:28.383894 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:54:28.383894 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:54:28.383959 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:54:28.384152 (+ 193us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:54:28.384153 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:54:28.384204 (+ 51us) client_negotiation.cc:769] Sending connection context
0808 16:54:28.384275 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 16:54:28.384376 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20}
I20260808 16:54:28.384862 29564 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.24.60" port: 46443 } has no permanent_uuid. Determining permanent_uuid...
I20260808 16:54:28.385186 29575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:28.385083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.60:46443 (local address 127.0.0.1:37720)
0808 16:54:28.385130 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:28.385164 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.60:46443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260808 16:54:28.385522 29575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:28.385429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.60:46443 (local address 127.0.0.1:37724)
0808 16:54:28.385473 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:28.385502 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.60:46443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260808 16:54:28.385607 29564 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.24.60:46443: Network error: Client connection negotiation failed: client connection to 127.8.24.60:46443: connect: Connection refused (error 111)
W20260808 16:54:28.394896 29694 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:54:28.395071 29694 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:54:28.395090 29694 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:54:28.396139 29694 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 16:54:28.396176 29694 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:54:28.396189 29694 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:54:28.396200 29694 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 16:54:28.396216 29694 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 16:54:28.397624 29694 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:45551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786207919384338-8288-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786207919384338-8288-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:45743,127.8.24.61:33015,127.8.24.60:46443
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.60.keytab
--principal=kudu/127.8.24.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.60:46443
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786207919384338-8288-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.8.24.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29694
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786207919384338-8288-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:54:28.397804 29694 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:54:28.397966 29694 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:54:28.402182 29701 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:54:28.402148 29706 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:54:28.402159 29699 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:54:28.402539 29694 server_base.cc:1034] running on GCE node
I20260808 16:54:28.402745 29694 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:54:28.402959 29694 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:54:28.404088 29694 hybrid_clock.cc:648] HybridClock initialized: now 1786208068404073 us; error 26 us; skew 500 ppm
Aug 08 16:54:28 dist-test-slave-6zbq krb5kdc[29425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208068, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:54:28.406173 29694 init.cc:377] Logged in from keytab as kudu/127.8.24.60@KRBTEST.COM (short username kudu)
I20260808 16:54:28.406572 29694 webserver.cc:460] Webserver started at http://127.8.24.60:38289/ using document root <none> and password file <none>
I20260808 16:54:28.406767 29694 fs_manager.cc:359] Metadata directory not provided
I20260808 16:54:28.406814 29694 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:54:28.406934 29694 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:54:28.407861 29694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786207919384338-8288-0/minicluster-data/master-2/data/instance:
uuid: "503a23b4b2f6447f9691b056ab140195"
format_stamp: "Formatted at 2026-08-08 16:54:28 on dist-test-slave-6zbq"
server_key: "ffeb5ec339c00f51cb8c51d4aac6b97c"
server_key_iv: "baf1c5ded3c23fc877f22010a8ebeddb"
server_key_version: "encryptionkey@0"
I20260808 16:54:28.408164 29694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786207919384338-8288-0/minicluster-data/master-2/wal/instance:
uuid: "503a23b4b2f6447f9691b056ab140195"
format_stamp: "Formatted at 2026-08-08 16:54:28 on dist-test-slave-6zbq"
server_key: "ffeb5ec339c00f51cb8c51d4aac6b97c"
server_key_iv: "baf1c5ded3c23fc877f22010a8ebeddb"
server_key_version: "encryptionkey@0"
I20260808 16:54:28.409394 29694 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 16:54:28.409976 29720 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:54:28.410122 29694 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 16:54:28.410198 29694 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786207919384338-8288-0/minicluster-data/master-2/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786207919384338-8288-0/minicluster-data/master-2/wal
uuid: "503a23b4b2f6447f9691b056ab140195"
format_stamp: "Formatted at 2026-08-08 16:54:28 on dist-test-slave-6zbq"
server_key: "ffeb5ec339c00f51cb8c51d4aac6b97c"
server_key_iv: "baf1c5ded3c23fc877f22010a8ebeddb"
server_key_version: "encryptionkey@0"
I20260808 16:54:28.410259 29694 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786207919384338-8288-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786207919384338-8288-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786207919384338-8288-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:54:28.416721 29564 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.24.60" port: 46443 } attempt: 1
I20260808 16:54:28.416991 29575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:28.416907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.60:46443 (local address 127.0.0.1:37732)
0808 16:54:28.416947 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:28.416970 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.60:46443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260808 16:54:28.417249 29575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:28.417179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.60:46443 (local address 127.0.0.1:37742)
0808 16:54:28.417212 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:28.417234 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.60:46443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260808 16:54:28.417317 29564 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.24.60:46443: Network error: Client connection negotiation failed: client connection to 127.8.24.60:46443: connect: Connection refused (error 111)
I20260808 16:54:28.419456 29694 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:54:28.421272 29694 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:54:28.421391 29694 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:54:28.424852 29694 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.60:46443
I20260808 16:54:28.424880 29804 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.60:46443 every 8 connection(s)
I20260808 16:54:28.425153 29694 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786207919384338-8288-0/minicluster-data/master-2/data/info.pb
I20260808 16:54:28.425410 29683 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.24.60" port: 46443 } attempt: 2
I20260808 16:54:28.426038 29806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:54:28.427407 29806 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.24.62" port: 45743 } has no permanent_uuid. Determining permanent_uuid...
I20260808 16:54:28.429255 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 29694
I20260808 16:54:28.429359 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786207919384338-8288-0/minicluster-data/master-2/wal/instance
I20260808 16:54:28.429569 8288 external_mini_cluster.cc:1428] Setting key d5c174e913ea257be1a67bfe80ec9356
Aug 08 16:54:28 dist-test-slave-6zbq krb5kdc[29425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208068, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.61@KRBTEST.COM for kudu/127.8.24.60@KRBTEST.COM
Aug 08 16:54:28 dist-test-slave-6zbq krb5kdc[29425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208068, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.60@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260808 16:54:28.434055 29685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:28.425656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.60:46443 (local address 127.0.0.1:37750)
0808 16:54:28.425710 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:28.425721 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 16:54:28.425768 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:54:28.426555 (+ 787us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:54:28.426556 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:54:28.426716 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:54:28.426751 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:54:28.426753 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:54:28.427788 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:54:28.427789 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:54:28.428503 (+ 714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:54:28.428505 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:54:28.429229 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:54:28.429230 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:54:28.429284 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:28.429733 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:54:28.429738 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:54:28.431044 (+ 1306us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:54:28.432952 (+ 1908us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:54:28.432954 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:54:28.432954 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:54:28.433120 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:54:28.433328 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:54:28.433329 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:54:28.433329 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:54:28.433387 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:54:28.433758 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:54:28.433759 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:54:28.433806 (+ 47us) client_negotiation.cc:769] Sending connection context
0808 16:54:28.433876 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 16:54:28.433964 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260808 16:54:28.434239 29808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:28.425777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37750 (local address 127.8.24.60:46443)
0808 16:54:28.425942 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0808 16:54:28.425949 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 16:54:28.425971 (+ 22us) server_negotiation.cc:407] Connection header received
0808 16:54:28.426387 (+ 416us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:54:28.426394 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:54:28.426448 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:54:28.426563 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:54:28.426842 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:28.427712 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:28.428583 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:28.429140 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:28.429250 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:28.431130 (+ 1880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:54:28.431138 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:54:28.431138 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:54:28.431150 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:54:28.432872 (+ 1722us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:54:28.433191 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:54:28.433194 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:54:28.433195 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:54:28.433270 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:54:28.433436 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:54:28.433437 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:54:28.433437 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:54:28.433692 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:54:28.433775 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 16:54:28.433918 (+ 143us) server_negotiation.cc:299] Negotiation successful
0808 16:54:28.434032 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":88,"threads_started":1}
Aug 08 16:54:28 dist-test-slave-6zbq krb5kdc[29425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208068, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.24.60@KRBTEST.COM
I20260808 16:54:28.435868 29577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:28.427972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41344 (local address 127.8.24.62:45743)
0808 16:54:28.428015 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 16:54:28.428017 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:54:28.428125 (+ 108us) server_negotiation.cc:407] Connection header received
0808 16:54:28.428185 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:54:28.428186 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:54:28.428194 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:54:28.428239 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:54:28.428695 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:28.429370 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:28.430637 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:28.430910 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:28.430980 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:28.433002 (+ 2022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:54:28.433004 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:54:28.433004 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:54:28.433011 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:54:28.434787 (+ 1776us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:54:28.435152 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:54:28.435153 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:54:28.435154 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:54:28.435211 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:54:28.435369 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:54:28.435370 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:54:28.435371 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:54:28.435472 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:54:28.435524 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 16:54:28.435716 (+ 192us) server_negotiation.cc:299] Negotiation successful
0808 16:54:28.435786 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 16:54:28.435884 29811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:28.427905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:45743 (local address 127.0.0.1:41344)
0808 16:54:28.428032 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:28.428070 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 16:54:28.428131 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:54:28.428263 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:54:28.428265 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:54:28.428545 (+ 280us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:54:28.428624 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:54:28.428627 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:54:28.429461 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:54:28.429462 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:54:28.430558 (+ 1096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:54:28.430561 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:54:28.431002 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:54:28.431003 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:54:28.431078 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:28.431517 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:54:28.431521 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:54:28.432896 (+ 1375us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:54:28.434853 (+ 1957us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:54:28.434856 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:54:28.434858 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:54:28.435084 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:54:28.435264 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:54:28.435264 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:54:28.435265 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:54:28.435321 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:54:28.435553 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:54:28.435559 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:54:28.435609 (+ 50us) client_negotiation.cc:769] Sending connection context
0808 16:54:28.435684 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 16:54:28.435785 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":68,"threads_started":1}
I20260808 16:54:28.436030 29683 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93ead526500149af99a55fe16fba1e75: Bootstrap starting.
I20260808 16:54:28.436311 29806 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.24.61" port: 33015 } has no permanent_uuid. Determining permanent_uuid...
I20260808 16:54:28.437129 29683 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 93ead526500149af99a55fe16fba1e75: Neither blocks nor log segments found. Creating new log.
I20260808 16:54:28.437438 29683 log.cc:824] T 00000000000000000000000000000000 P 93ead526500149af99a55fe16fba1e75: Log is configured to *not* fsync() on all Append() calls
I20260808 16:54:28.438248 29683 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93ead526500149af99a55fe16fba1e75: No bootstrap required, opened a new log
I20260808 16:54:28.439406 29813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:28.429898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37756 (local address 127.8.24.60:46443)
0808 16:54:28.430428 (+ 530us) server_negotiation.cc:206] Beginning negotiation
0808 16:54:28.430430 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:54:28.430437 (+ 7us) server_negotiation.cc:407] Connection header received
0808 16:54:28.430454 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:54:28.430454 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:54:28.430464 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:54:28.430518 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:54:28.431195 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:28.431828 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:28.432997 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:28.433487 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:28.433550 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:28.435967 (+ 2417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:54:28.435968 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:54:28.435968 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:54:28.435975 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:54:28.438093 (+ 2118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:54:28.438491 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:54:28.438492 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:54:28.438493 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:54:28.438549 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:54:28.438855 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:54:28.438855 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:54:28.438856 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:54:28.438976 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:54:28.439096 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0808 16:54:28.439219 (+ 123us) server_negotiation.cc:299] Negotiation successful
0808 16:54:28.439291 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":510,"thread_start_us":72,"threads_started":1}
Aug 08 16:54:28 dist-test-slave-6zbq krb5kdc[29425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208068, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.60@KRBTEST.COM for kudu/127.8.24.61@KRBTEST.COM
I20260808 16:54:28.440186 29683 raft_consensus.cc:348] T 00000000000000000000000000000000 P 93ead526500149af99a55fe16fba1e75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a79a0a7cfd94418be2c577f70bb56c6" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 45743 } } peers { permanent_uuid: "93ead526500149af99a55fe16fba1e75" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 33015 } } peers { permanent_uuid: "503a23b4b2f6447f9691b056ab140195" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 46443 } }
I20260808 16:54:28.440346 29683 raft_consensus.cc:374] T 00000000000000000000000000000000 P 93ead526500149af99a55fe16fba1e75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:54:28.440441 29683 raft_consensus.cc:729] T 00000000000000000000000000000000 P 93ead526500149af99a55fe16fba1e75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93ead526500149af99a55fe16fba1e75, State: Initialized, Role: FOLLOWER
I20260808 16:54:28.440572 29683 consensus_queue.cc:260] T 00000000000000000000000000000000 P 93ead526500149af99a55fe16fba1e75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a79a0a7cfd94418be2c577f70bb56c6" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 45743 } } peers { permanent_uuid: "93ead526500149af99a55fe16fba1e75" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 33015 } } peers { permanent_uuid: "503a23b4b2f6447f9691b056ab140195" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 46443 } }
I20260808 16:54:28.440866 29683 sys_catalog.cc:564] T 00000000000000000000000000000000 P 93ead526500149af99a55fe16fba1e75 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 16:54:28.442553 29811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:28.436508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.61:33015 (local address 127.0.0.1:40606)
0808 16:54:28.436569 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:28.436577 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 16:54:28.436613 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:54:28.436717 (+ 104us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:54:28.436718 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:54:28.436835 (+ 117us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:54:28.436878 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:54:28.436879 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:54:28.437485 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:54:28.437486 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:54:28.438068 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:54:28.438071 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:54:28.438444 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:54:28.438445 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:54:28.438509 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:28.438827 (+ 318us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:54:28.438830 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:54:28.440126 (+ 1296us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:54:28.441558 (+ 1432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:54:28.441559 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:54:28.441559 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:54:28.441720 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:54:28.441902 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:54:28.441903 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:54:28.441903 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:54:28.441962 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:54:28.442268 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:54:28.442269 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:54:28.442312 (+ 43us) client_negotiation.cc:769] Sending connection context
0808 16:54:28.442397 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 16:54:28.442476 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
W20260808 16:54:28.442792 29832 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 93ead526500149af99a55fe16fba1e75: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 16:54:28.442858 29832 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 16:54:28.443159 29690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:28.436569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40606 (local address 127.8.24.61:33015)
0808 16:54:28.436601 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0808 16:54:28.436603 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:54:28.436610 (+ 7us) server_negotiation.cc:407] Connection header received
0808 16:54:28.436667 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:54:28.436668 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:54:28.436676 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:54:28.436715 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:54:28.436940 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:28.437417 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:28.438139 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:28.438386 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:28.438444 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:28.440190 (+ 1746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:54:28.440191 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:54:28.440192 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:54:28.440198 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:54:28.441468 (+ 1270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:54:28.441784 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:54:28.441785 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:54:28.441785 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:54:28.441842 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:54:28.442005 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:54:28.442006 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:54:28.442006 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:54:28.442149 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:54:28.442211 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 16:54:28.443014 (+ 803us) server_negotiation.cc:299] Negotiation successful
0808 16:54:28.443088 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 16:54:28.442909 29582 master_runner.cc:186] Error getting master registration for 127.8.24.62:45743: OK, instance_id { permanent_uuid: "2a79a0a7cfd94418be2c577f70bb56c6" instance_seqno: 1786208068153494 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 16:54:28.445025 29582 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 16:54:28.442248 29815 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93ead526500149af99a55fe16fba1e75 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a79a0a7cfd94418be2c577f70bb56c6" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 45743 } } peers { permanent_uuid: "93ead526500149af99a55fe16fba1e75" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 33015 } } peers { permanent_uuid: "503a23b4b2f6447f9691b056ab140195" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 46443 } } }
I20260808 16:54:28.445292 29815 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93ead526500149af99a55fe16fba1e75 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 16:54:28.445340 29806 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.24.60" port: 46443 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 16:54:28 dist-test-slave-6zbq krb5kdc[29425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208068, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260808 16:54:28.448908 29577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:28.440638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41356 (local address 127.8.24.62:45743)
0808 16:54:28.440671 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0808 16:54:28.440682 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 16:54:28.440795 (+ 113us) server_negotiation.cc:407] Connection header received
0808 16:54:28.441137 (+ 342us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:54:28.441139 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:54:28.441148 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:54:28.441174 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:54:28.441768 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:28.442403 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:28.443269 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:28.443531 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:28.443567 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:28.445951 (+ 2384us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:54:28.445952 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:54:28.445952 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:54:28.445958 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:54:28.447765 (+ 1807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:54:28.448073 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:54:28.448074 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:54:28.448074 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:54:28.448126 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:54:28.448310 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:54:28.448311 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:54:28.448311 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:54:28.448424 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:54:28.448559 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0808 16:54:28.448769 (+ 210us) server_negotiation.cc:299] Negotiation successful
0808 16:54:28.448834 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
Aug 08 16:54:28 dist-test-slave-6zbq krb5kdc[29425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208068, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.60@KRBTEST.COM for kudu/127.8.24.60@KRBTEST.COM
I20260808 16:54:28.451828 29811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:28.445522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.60:46443 (local address 127.0.0.1:37764)
0808 16:54:28.445581 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:28.445589 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 16:54:28.445620 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:54:28.445736 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:54:28.445737 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:54:28.445861 (+ 124us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:54:28.445891 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:54:28.445892 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:54:28.446743 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:54:28.446744 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:54:28.447216 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:54:28.447218 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:54:28.447673 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:54:28.447674 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:54:28.447731 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:28.448217 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:54:28.448221 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:54:28.449455 (+ 1234us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:54:28.450996 (+ 1541us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:54:28.450996 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:54:28.450997 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:54:28.451140 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:54:28.451293 (+ 153us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:54:28.451293 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:54:28.451294 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:54:28.451343 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:54:28.451552 (+ 209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:54:28.451553 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:54:28.451587 (+ 34us) client_negotiation.cc:769] Sending connection context
0808 16:54:28.451646 (+ 59us) client_negotiation.cc:240] Negotiation successful
0808 16:54:28.451741 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260808 16:54:28.451884 29813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:28.445593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37764 (local address 127.8.24.60:46443)
0808 16:54:28.445629 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0808 16:54:28.445630 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 16:54:28.445639 (+ 9us) server_negotiation.cc:407] Connection header received
0808 16:54:28.445667 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:54:28.445668 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:54:28.445677 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:54:28.445732 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:54:28.445960 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:28.446673 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:28.447288 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:28.447586 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:28.447653 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:28.449642 (+ 1989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:54:28.449643 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:54:28.449643 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:54:28.449650 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:54:28.450925 (+ 1275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:54:28.451197 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:54:28.451198 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:54:28.451198 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:54:28.451243 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:54:28.451390 (+ 147us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:54:28.451390 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:54:28.451391 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:54:28.451499 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:54:28.451551 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 16:54:28.451671 (+ 120us) server_negotiation.cc:299] Negotiation successful
0808 16:54:28.451791 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":59,"server-negotiator.queue_time_us":21}
I20260808 16:54:28.452833 29806 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 503a23b4b2f6447f9691b056ab140195: Bootstrap starting.
I20260808 16:54:28.453434 29806 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 503a23b4b2f6447f9691b056ab140195: Neither blocks nor log segments found. Creating new log.
I20260808 16:54:28.453626 29806 log.cc:824] T 00000000000000000000000000000000 P 503a23b4b2f6447f9691b056ab140195: Log is configured to *not* fsync() on all Append() calls
I20260808 16:54:28.454305 29806 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 503a23b4b2f6447f9691b056ab140195: No bootstrap required, opened a new log
Aug 08 16:54:28 dist-test-slave-6zbq krb5kdc[29425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208068, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.24.61@KRBTEST.COM
I20260808 16:54:28.456116 29806 raft_consensus.cc:348] T 00000000000000000000000000000000 P 503a23b4b2f6447f9691b056ab140195 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a79a0a7cfd94418be2c577f70bb56c6" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 45743 } } peers { permanent_uuid: "93ead526500149af99a55fe16fba1e75" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 33015 } } peers { permanent_uuid: "503a23b4b2f6447f9691b056ab140195" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 46443 } }
I20260808 16:54:28.456286 29806 raft_consensus.cc:374] T 00000000000000000000000000000000 P 503a23b4b2f6447f9691b056ab140195 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:54:28.456368 29806 raft_consensus.cc:729] T 00000000000000000000000000000000 P 503a23b4b2f6447f9691b056ab140195 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 503a23b4b2f6447f9691b056ab140195, State: Initialized, Role: FOLLOWER
I20260808 16:54:28.456502 29806 consensus_queue.cc:260] T 00000000000000000000000000000000 P 503a23b4b2f6447f9691b056ab140195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a79a0a7cfd94418be2c577f70bb56c6" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 45743 } } peers { permanent_uuid: "93ead526500149af99a55fe16fba1e75" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 33015 } } peers { permanent_uuid: "503a23b4b2f6447f9691b056ab140195" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 46443 } }
I20260808 16:54:28.456785 29690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:28.449642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40608 (local address 127.8.24.61:33015)
0808 16:54:28.449682 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 16:54:28.449683 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 16:54:28.449690 (+ 7us) server_negotiation.cc:407] Connection header received
0808 16:54:28.449710 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:54:28.449710 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:54:28.449718 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:54:28.449778 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:54:28.450269 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:28.450972 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:28.452176 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:28.452449 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:28.452489 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:28.454750 (+ 2261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:54:28.454751 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:54:28.454752 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:54:28.454758 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:54:28.455876 (+ 1118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:54:28.456151 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:54:28.456152 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:54:28.456152 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:54:28.456208 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:54:28.456360 (+ 152us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:54:28.456360 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:54:28.456361 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:54:28.456469 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:54:28.456574 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 16:54:28.456648 (+ 74us) server_negotiation.cc:299] Negotiation successful
0808 16:54:28.456711 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 16:54:28.456833 29806 sys_catalog.cc:564] T 00000000000000000000000000000000 P 503a23b4b2f6447f9691b056ab140195 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 16:54:28.458027 8288 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 16:54:28.458153 29838 sys_catalog.cc:455] T 00000000000000000000000000000000 P 503a23b4b2f6447f9691b056ab140195 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a79a0a7cfd94418be2c577f70bb56c6" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 45743 } } peers { permanent_uuid: "93ead526500149af99a55fe16fba1e75" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 33015 } } peers { permanent_uuid: "503a23b4b2f6447f9691b056ab140195" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 46443 } } }
I20260808 16:54:28.458231 29838 sys_catalog.cc:458] T 00000000000000000000000000000000 P 503a23b4b2f6447f9691b056ab140195 [sys.catalog]: This master's current role is: FOLLOWER
W20260808 16:54:28.458843 29859 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 503a23b4b2f6447f9691b056ab140195: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 16:54:28.458886 29859 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 16:54:28.460052 29694 master_runner.cc:186] Error getting master registration for 127.8.24.62:45743: OK, instance_id { permanent_uuid: "2a79a0a7cfd94418be2c577f70bb56c6" instance_seqno: 1786208068153494 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 16:54:28.462093 29694 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 16:54:28.465646 29813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:28.459563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37766 (local address 127.8.24.60:46443)
0808 16:54:28.459598 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0808 16:54:28.459600 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:54:28.459608 (+ 8us) server_negotiation.cc:407] Connection header received
0808 16:54:28.460096 (+ 488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:54:28.460097 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:54:28.460106 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:54:28.460133 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:54:28.460797 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:28.461470 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:28.462396 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:28.462590 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:28.462643 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:28.463640 (+ 997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:54:28.463641 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:54:28.463641 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:54:28.463647 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:54:28.464765 (+ 1118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:54:28.465042 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:54:28.465043 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:54:28.465043 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:54:28.465092 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:54:28.465245 (+ 153us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:54:28.465245 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:54:28.465245 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:54:28.465337 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:54:28.465424 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 16:54:28.465510 (+ 86us) server_negotiation.cc:299] Negotiation successful
0808 16:54:28.465573 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 16:54:28.474167 29690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:28.459712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40624 (local address 127.8.24.61:33015)
0808 16:54:28.459741 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0808 16:54:28.459743 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:54:28.466038 (+ 6295us) server_negotiation.cc:407] Connection header received
0808 16:54:28.466128 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:54:28.466129 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:54:28.466138 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:54:28.466192 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:54:28.466694 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:28.467338 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:28.468221 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:28.468499 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:28.468533 (+ 34us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:28.470046 (+ 1513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:54:28.470048 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:54:28.470048 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:54:28.470055 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:54:28.472934 (+ 2879us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:54:28.473328 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:54:28.473329 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:54:28.473329 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:54:28.473391 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:54:28.473614 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:54:28.473615 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:54:28.473615 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:54:28.473722 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:54:28.473778 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 16:54:28.474012 (+ 234us) server_negotiation.cc:299] Negotiation successful
0808 16:54:28.474082 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260808 16:54:28.474167 29577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:28.459877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41372 (local address 127.8.24.62:45743)
0808 16:54:28.459906 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0808 16:54:28.459908 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:54:28.466582 (+ 6674us) server_negotiation.cc:407] Connection header received
0808 16:54:28.466631 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:54:28.466632 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:54:28.466642 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:54:28.466689 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:54:28.467312 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:28.467964 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:28.468842 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:28.469103 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:28.469143 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:28.470399 (+ 1256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:54:28.470400 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:54:28.470401 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:54:28.470407 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:54:28.472854 (+ 2447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:54:28.473329 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:54:28.473330 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:54:28.473330 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:54:28.473389 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:54:28.473602 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:54:28.473603 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:54:28.473603 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:54:28.473715 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:54:28.473778 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 16:54:28.474012 (+ 234us) server_negotiation.cc:299] Negotiation successful
0808 16:54:28.474082 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260808 16:54:28.492440 29564 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.24.60" port: 46443 } attempt: 2
Aug 08 16:54:28 dist-test-slave-6zbq krb5kdc[29425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208068, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for kudu/127.8.24.60@KRBTEST.COM
I20260808 16:54:28.498175 29813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:28.492674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37772 (local address 127.8.24.60:46443)
0808 16:54:28.492707 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0808 16:54:28.492708 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 16:54:28.492716 (+ 8us) server_negotiation.cc:407] Connection header received
0808 16:54:28.492769 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:54:28.492770 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:54:28.492778 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:54:28.492822 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:54:28.493041 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:28.493464 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:28.494067 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:28.494243 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:28.494294 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:28.496142 (+ 1848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:54:28.496143 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:54:28.496144 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:54:28.496150 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:54:28.497285 (+ 1135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:54:28.497562 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:54:28.497563 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:54:28.497563 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:54:28.497610 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:54:28.497764 (+ 154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:54:28.497765 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:54:28.497765 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:54:28.497855 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:54:28.497906 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 16:54:28.498032 (+ 126us) server_negotiation.cc:299] Negotiation successful
0808 16:54:28.498090 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260808 16:54:28.498185 29575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:28.492634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.60:46443 (local address 127.0.0.1:37772)
0808 16:54:28.492683 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:28.492692 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 16:54:28.492727 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:54:28.492830 (+ 103us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:54:28.492831 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:54:28.492952 (+ 121us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:54:28.492981 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:54:28.492983 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:54:28.493536 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:54:28.493537 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:54:28.494000 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:54:28.494001 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:54:28.494299 (+ 298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:54:28.494300 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:54:28.494376 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:28.494951 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:54:28.494956 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:54:28.496070 (+ 1114us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:54:28.497356 (+ 1286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:54:28.497357 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:54:28.497358 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:54:28.497505 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:54:28.497662 (+ 157us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:54:28.497662 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:54:28.497663 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:54:28.497716 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:54:28.497907 (+ 191us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:54:28.497908 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:54:28.497947 (+ 39us) client_negotiation.cc:769] Sending connection context
0808 16:54:28.498005 (+ 58us) client_negotiation.cc:240] Negotiation successful
0808 16:54:28.498089 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260808 16:54:28.499392 29564 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a79a0a7cfd94418be2c577f70bb56c6: Bootstrap starting.
I20260808 16:54:28.499975 29564 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2a79a0a7cfd94418be2c577f70bb56c6: Neither blocks nor log segments found. Creating new log.
I20260808 16:54:28.500240 29564 log.cc:824] T 00000000000000000000000000000000 P 2a79a0a7cfd94418be2c577f70bb56c6: Log is configured to *not* fsync() on all Append() calls
I20260808 16:54:28.500908 29564 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a79a0a7cfd94418be2c577f70bb56c6: No bootstrap required, opened a new log
I20260808 16:54:28.502295 29564 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2a79a0a7cfd94418be2c577f70bb56c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a79a0a7cfd94418be2c577f70bb56c6" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 45743 } } peers { permanent_uuid: "93ead526500149af99a55fe16fba1e75" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 33015 } } peers { permanent_uuid: "503a23b4b2f6447f9691b056ab140195" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 46443 } }
I20260808 16:54:28.502430 29564 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2a79a0a7cfd94418be2c577f70bb56c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:54:28.502491 29564 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2a79a0a7cfd94418be2c577f70bb56c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a79a0a7cfd94418be2c577f70bb56c6, State: Initialized, Role: FOLLOWER
I20260808 16:54:28.502610 29564 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2a79a0a7cfd94418be2c577f70bb56c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a79a0a7cfd94418be2c577f70bb56c6" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 45743 } } peers { permanent_uuid: "93ead526500149af99a55fe16fba1e75" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 33015 } } peers { permanent_uuid: "503a23b4b2f6447f9691b056ab140195" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 46443 } }
I20260808 16:54:28.502897 29564 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2a79a0a7cfd94418be2c577f70bb56c6 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 16:54:28.503559 29875 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a79a0a7cfd94418be2c577f70bb56c6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a79a0a7cfd94418be2c577f70bb56c6" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 45743 } } peers { permanent_uuid: "93ead526500149af99a55fe16fba1e75" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 33015 } } peers { permanent_uuid: "503a23b4b2f6447f9691b056ab140195" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 46443 } } }
I20260808 16:54:28.503711 29875 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a79a0a7cfd94418be2c577f70bb56c6 [sys.catalog]: This master's current role is: FOLLOWER
W20260808 16:54:28.504588 29893 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2a79a0a7cfd94418be2c577f70bb56c6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 16:54:28.504689 29893 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 16:54:28.506672 29449 master_runner.cc:263] No leader master found from master 2a79a0a7cfd94418be2c577f70bb56c6
I20260808 16:54:28.506717 29449 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260808 16:54:28.535740 29609 tablet.cc:2369] T 00000000000000000000000000000000 P 93ead526500149af99a55fe16fba1e75: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 16:54:28.651611 29487 tablet.cc:2369] T 00000000000000000000000000000000 P 2a79a0a7cfd94418be2c577f70bb56c6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 16:54:28.671876 29729 tablet.cc:2369] T 00000000000000000000000000000000 P 503a23b4b2f6447f9691b056ab140195: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 16:54:28.683328 29815 raft_consensus.cc:482] T 00000000000000000000000000000000 P 93ead526500149af99a55fe16fba1e75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 16:54:28.683413 29815 raft_consensus.cc:504] T 00000000000000000000000000000000 P 93ead526500149af99a55fe16fba1e75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a79a0a7cfd94418be2c577f70bb56c6" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 45743 } } peers { permanent_uuid: "93ead526500149af99a55fe16fba1e75" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 33015 } } peers { permanent_uuid: "503a23b4b2f6447f9691b056ab140195" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 46443 } }
I20260808 16:54:28.683617 29815 leader_election.cc:288] T 00000000000000000000000000000000 P 93ead526500149af99a55fe16fba1e75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2a79a0a7cfd94418be2c577f70bb56c6 (127.8.24.62:45743), 503a23b4b2f6447f9691b056ab140195 (127.8.24.60:46443)
I20260808 16:54:28.683837 29522 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "93ead526500149af99a55fe16fba1e75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a79a0a7cfd94418be2c577f70bb56c6" is_pre_election: true
I20260808 16:54:28.683840 29768 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "93ead526500149af99a55fe16fba1e75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "503a23b4b2f6447f9691b056ab140195" is_pre_election: true
I20260808 16:54:28.683956 29768 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 503a23b4b2f6447f9691b056ab140195 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 93ead526500149af99a55fe16fba1e75 in term 0.
I20260808 16:54:28.683959 29522 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2a79a0a7cfd94418be2c577f70bb56c6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 93ead526500149af99a55fe16fba1e75 in term 0.
I20260808 16:54:28.684131 29607 leader_election.cc:302] T 00000000000000000000000000000000 P 93ead526500149af99a55fe16fba1e75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a79a0a7cfd94418be2c577f70bb56c6, 93ead526500149af99a55fe16fba1e75; no voters:
I20260808 16:54:28.684297 29815 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 93ead526500149af99a55fe16fba1e75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 16:54:28.684352 29815 raft_consensus.cc:482] T 00000000000000000000000000000000 P 93ead526500149af99a55fe16fba1e75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 16:54:28.684367 29815 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 93ead526500149af99a55fe16fba1e75 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:54:28.685029 29815 raft_consensus.cc:504] T 00000000000000000000000000000000 P 93ead526500149af99a55fe16fba1e75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a79a0a7cfd94418be2c577f70bb56c6" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 45743 } } peers { permanent_uuid: "93ead526500149af99a55fe16fba1e75" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 33015 } } peers { permanent_uuid: "503a23b4b2f6447f9691b056ab140195" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 46443 } }
I20260808 16:54:28.685153 29815 leader_election.cc:288] T 00000000000000000000000000000000 P 93ead526500149af99a55fe16fba1e75 [CANDIDATE]: Term 1 election: Requested vote from peers 2a79a0a7cfd94418be2c577f70bb56c6 (127.8.24.62:45743), 503a23b4b2f6447f9691b056ab140195 (127.8.24.60:46443)
I20260808 16:54:28.685282 29522 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "93ead526500149af99a55fe16fba1e75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a79a0a7cfd94418be2c577f70bb56c6"
I20260808 16:54:28.685297 29768 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "93ead526500149af99a55fe16fba1e75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "503a23b4b2f6447f9691b056ab140195"
I20260808 16:54:28.685357 29768 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 503a23b4b2f6447f9691b056ab140195 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:54:28.685351 29522 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2a79a0a7cfd94418be2c577f70bb56c6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:54:28.685958 29522 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2a79a0a7cfd94418be2c577f70bb56c6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93ead526500149af99a55fe16fba1e75 in term 1.
I20260808 16:54:28.685999 29768 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 503a23b4b2f6447f9691b056ab140195 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93ead526500149af99a55fe16fba1e75 in term 1.
I20260808 16:54:28.686131 29602 leader_election.cc:302] T 00000000000000000000000000000000 P 93ead526500149af99a55fe16fba1e75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2a79a0a7cfd94418be2c577f70bb56c6, 503a23b4b2f6447f9691b056ab140195, 93ead526500149af99a55fe16fba1e75; no voters:
I20260808 16:54:28.686244 29815 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 93ead526500149af99a55fe16fba1e75 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:54:28.686432 29815 raft_consensus.cc:686] T 00000000000000000000000000000000 P 93ead526500149af99a55fe16fba1e75 [term 1 LEADER]: Becoming Leader. State: Replica: 93ead526500149af99a55fe16fba1e75, State: Running, Role: LEADER
I20260808 16:54:28.686517 29815 consensus_queue.cc:237] T 00000000000000000000000000000000 P 93ead526500149af99a55fe16fba1e75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a79a0a7cfd94418be2c577f70bb56c6" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 45743 } } peers { permanent_uuid: "93ead526500149af99a55fe16fba1e75" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 33015 } } peers { permanent_uuid: "503a23b4b2f6447f9691b056ab140195" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 46443 } }
I20260808 16:54:28.686873 29898 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93ead526500149af99a55fe16fba1e75 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 93ead526500149af99a55fe16fba1e75. Latest consensus state: current_term: 1 leader_uuid: "93ead526500149af99a55fe16fba1e75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a79a0a7cfd94418be2c577f70bb56c6" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 45743 } } peers { permanent_uuid: "93ead526500149af99a55fe16fba1e75" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 33015 } } peers { permanent_uuid: "503a23b4b2f6447f9691b056ab140195" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 46443 } } }
I20260808 16:54:28.686961 29898 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93ead526500149af99a55fe16fba1e75 [sys.catalog]: This master's current role is: LEADER
I20260808 16:54:28.687148 29901 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 16:54:28.687333 29901 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 16:54:28.688542 29768 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 503a23b4b2f6447f9691b056ab140195 [term 1 FOLLOWER]: Refusing update from remote peer 93ead526500149af99a55fe16fba1e75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:54:28.688534 29522 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2a79a0a7cfd94418be2c577f70bb56c6 [term 1 FOLLOWER]: Refusing update from remote peer 93ead526500149af99a55fe16fba1e75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:54:28.688846 29898 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 93ead526500149af99a55fe16fba1e75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "503a23b4b2f6447f9691b056ab140195" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 46443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:54:28.688964 29815 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 93ead526500149af99a55fe16fba1e75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a79a0a7cfd94418be2c577f70bb56c6" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 45743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:54:28.689997 29875 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a79a0a7cfd94418be2c577f70bb56c6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 93ead526500149af99a55fe16fba1e75. Latest consensus state: current_term: 1 leader_uuid: "93ead526500149af99a55fe16fba1e75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a79a0a7cfd94418be2c577f70bb56c6" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 45743 } } peers { permanent_uuid: "93ead526500149af99a55fe16fba1e75" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 33015 } } peers { permanent_uuid: "503a23b4b2f6447f9691b056ab140195" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 46443 } } }
I20260808 16:54:28.690025 29838 sys_catalog.cc:455] T 00000000000000000000000000000000 P 503a23b4b2f6447f9691b056ab140195 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 93ead526500149af99a55fe16fba1e75. Latest consensus state: current_term: 1 leader_uuid: "93ead526500149af99a55fe16fba1e75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a79a0a7cfd94418be2c577f70bb56c6" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 45743 } } peers { permanent_uuid: "93ead526500149af99a55fe16fba1e75" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 33015 } } peers { permanent_uuid: "503a23b4b2f6447f9691b056ab140195" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 46443 } } }
I20260808 16:54:28.690083 29875 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a79a0a7cfd94418be2c577f70bb56c6 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 16:54:28.690099 29838 sys_catalog.cc:458] T 00000000000000000000000000000000 P 503a23b4b2f6447f9691b056ab140195 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 16:54:28.691032 29898 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93ead526500149af99a55fe16fba1e75 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "93ead526500149af99a55fe16fba1e75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a79a0a7cfd94418be2c577f70bb56c6" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 45743 } } peers { permanent_uuid: "93ead526500149af99a55fe16fba1e75" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 33015 } } peers { permanent_uuid: "503a23b4b2f6447f9691b056ab140195" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 46443 } } }
I20260808 16:54:28.691108 29898 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93ead526500149af99a55fe16fba1e75 [sys.catalog]: This master's current role is: LEADER
I20260808 16:54:28.691151 29898 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93ead526500149af99a55fe16fba1e75 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "93ead526500149af99a55fe16fba1e75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a79a0a7cfd94418be2c577f70bb56c6" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 45743 } } peers { permanent_uuid: "93ead526500149af99a55fe16fba1e75" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 33015 } } peers { permanent_uuid: "503a23b4b2f6447f9691b056ab140195" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 46443 } } }
I20260808 16:54:28.691190 29898 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93ead526500149af99a55fe16fba1e75 [sys.catalog]: This master's current role is: LEADER
I20260808 16:54:28.691228 29875 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a79a0a7cfd94418be2c577f70bb56c6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "93ead526500149af99a55fe16fba1e75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a79a0a7cfd94418be2c577f70bb56c6" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 45743 } } peers { permanent_uuid: "93ead526500149af99a55fe16fba1e75" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 33015 } } peers { permanent_uuid: "503a23b4b2f6447f9691b056ab140195" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 46443 } } }
I20260808 16:54:28.691303 29875 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a79a0a7cfd94418be2c577f70bb56c6 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 16:54:28.691316 29838 sys_catalog.cc:455] T 00000000000000000000000000000000 P 503a23b4b2f6447f9691b056ab140195 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "93ead526500149af99a55fe16fba1e75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a79a0a7cfd94418be2c577f70bb56c6" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 45743 } } peers { permanent_uuid: "93ead526500149af99a55fe16fba1e75" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 33015 } } peers { permanent_uuid: "503a23b4b2f6447f9691b056ab140195" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 46443 } } }
I20260808 16:54:28.691385 29838 sys_catalog.cc:458] T 00000000000000000000000000000000 P 503a23b4b2f6447f9691b056ab140195 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 16:54:28.691390 29903 mvcc.cc:205] Tried to move back new op lower bound from 7316308249346170880 to 7316308249340407808. Current Snapshot: MvccSnapshot[applied={T|T < 7316308249346170880}]
I20260808 16:54:28.691522 29904 mvcc.cc:205] Tried to move back new op lower bound from 7316308249346170880 to 7316308249340407808. Current Snapshot: MvccSnapshot[applied={T|T < 7316308249346170880}]
I20260808 16:54:28.691692 29901 catalog_manager.cc:1347] Generated new cluster ID: c4e3a862d4c3439faff590c92c7abd84
I20260808 16:54:28.691746 29901 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 16:54:28.699399 29901 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 16:54:28.700042 29901 catalog_manager.cc:1504] Loading token signing keys...
I20260808 16:54:28.706933 29901 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 93ead526500149af99a55fe16fba1e75: Generated new TSK 0
I20260808 16:54:28.707144 29901 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 16:54:29.459275 29859 catalog_manager.cc:1259] Loaded cluster ID: c4e3a862d4c3439faff590c92c7abd84
I20260808 16:54:29.459342 29859 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 503a23b4b2f6447f9691b056ab140195: loading cluster ID for follower catalog manager: success
I20260808 16:54:29.460134 29859 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 503a23b4b2f6447f9691b056ab140195: acquiring CA information for follower catalog manager: success
I20260808 16:54:29.460384 29859 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 503a23b4b2f6447f9691b056ab140195: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 16:54:29.505113 29893 catalog_manager.cc:1259] Loaded cluster ID: c4e3a862d4c3439faff590c92c7abd84
I20260808 16:54:29.505170 29893 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 2a79a0a7cfd94418be2c577f70bb56c6: loading cluster ID for follower catalog manager: success
I20260808 16:54:29.505875 29893 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 2a79a0a7cfd94418be2c577f70bb56c6: acquiring CA information for follower catalog manager: success
I20260808 16:54:29.506122 29893 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 2a79a0a7cfd94418be2c577f70bb56c6: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 16:54:29.751968 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 29449
I20260808 16:54:29.765137 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 29582
I20260808 16:54:29.771987 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 29694
2026-08-08T16:54:29Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (2954 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786207919384338-8288-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 16:54:29 dist-test-slave-6zbq krb5kdc[29921](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 16:54:29 dist-test-slave-6zbq krb5kdc[29921](info): set up 2 sockets
Aug 08 16:54:29 dist-test-slave-6zbq krb5kdc[29921](info): commencing operation
krb5kdc: starting...
W20260808 16:54:30.886791 8288 mini_kdc.cc:121] Time spent starting KDC: real 1.092s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786207919384338-8288-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786207919384338-8288-0/krb5kdc/test-user.keytab.
Aug 08 16:54:30 dist-test-slave-6zbq krb5kdc[29921](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208070, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T16:54:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T16:54:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.24.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
I20260808 16:54:30.992626 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786207919384338-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786207919384338-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786207919384338-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:44301
--webserver_interface=127.8.24.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:38319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:44301
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:54:31.069519 29946 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:54:31.069692 29946 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:54:31.069721 29946 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:54:31.070812 29946 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 16:54:31.070851 29946 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:54:31.070870 29946 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:54:31.070886 29946 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 16:54:31.070904 29946 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 16:54:31.072312 29946 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:38319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786207919384338-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786207919384338-8288-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:44301
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:44301
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.24.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29946
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786207919384338-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:54:31.072532 29946 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:54:31.072731 29946 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:54:31.077090 29952 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:54:31.077100 29955 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:54:31.077121 29953 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:54:31.077224 29946 server_base.cc:1034] running on GCE node
I20260808 16:54:31.077579 29946 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:54:31.077836 29946 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:54:31.078966 29946 hybrid_clock.cc:648] HybridClock initialized: now 1786208071078947 us; error 31 us; skew 500 ppm
Aug 08 16:54:31 dist-test-slave-6zbq krb5kdc[29921](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208071, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:54:31.081199 29946 init.cc:377] Logged in from keytab as kudu/127.8.24.62@KRBTEST.COM (short username kudu)
I20260808 16:54:31.081594 29946 webserver.cc:460] Webserver started at http://127.8.24.62:42523/ using document root <none> and password file <none>
I20260808 16:54:31.081800 29946 fs_manager.cc:359] Metadata directory not provided
I20260808 16:54:31.081852 29946 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:54:31.081969 29946 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:54:31.082944 29946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786207919384338-8288-0/minicluster-data/master-0/data/instance:
uuid: "558963c9265449d693b9f291d37f44c9"
format_stamp: "Formatted at 2026-08-08 16:54:31 on dist-test-slave-6zbq"
server_key: "427cdfc5f1cd992b8300ca585b1a84b0"
server_key_iv: "d7b60bdb4f4e56d134a3f161161d5363"
server_key_version: "encryptionkey@0"
I20260808 16:54:31.083248 29946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786207919384338-8288-0/minicluster-data/master-0/wal/instance:
uuid: "558963c9265449d693b9f291d37f44c9"
format_stamp: "Formatted at 2026-08-08 16:54:31 on dist-test-slave-6zbq"
server_key: "427cdfc5f1cd992b8300ca585b1a84b0"
server_key_iv: "d7b60bdb4f4e56d134a3f161161d5363"
server_key_version: "encryptionkey@0"
I20260808 16:54:31.084367 29946 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 16:54:31.084952 29966 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:54:31.085107 29946 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 16:54:31.085192 29946 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786207919384338-8288-0/minicluster-data/master-0/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786207919384338-8288-0/minicluster-data/master-0/wal
uuid: "558963c9265449d693b9f291d37f44c9"
format_stamp: "Formatted at 2026-08-08 16:54:31 on dist-test-slave-6zbq"
server_key: "427cdfc5f1cd992b8300ca585b1a84b0"
server_key_iv: "d7b60bdb4f4e56d134a3f161161d5363"
server_key_version: "encryptionkey@0"
I20260808 16:54:31.085250 29946 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786207919384338-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786207919384338-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786207919384338-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:54:31.095229 29946 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:54:31.097096 29946 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:54:31.097223 29946 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:54:31.100832 29946 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.62:44301
I20260808 16:54:31.100860 30065 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.62:44301 every 8 connection(s)
I20260808 16:54:31.101143 29946 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
I20260808 16:54:31.101852 30066 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:54:31.103785 30066 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 558963c9265449d693b9f291d37f44c9: Bootstrap starting.
I20260808 16:54:31.104259 30066 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 558963c9265449d693b9f291d37f44c9: Neither blocks nor log segments found. Creating new log.
I20260808 16:54:31.104475 30066 log.cc:824] T 00000000000000000000000000000000 P 558963c9265449d693b9f291d37f44c9: Log is configured to *not* fsync() on all Append() calls
I20260808 16:54:31.105041 30066 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 558963c9265449d693b9f291d37f44c9: No bootstrap required, opened a new log
I20260808 16:54:31.106448 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 29946
I20260808 16:54:31.106534 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786207919384338-8288-0/minicluster-data/master-0/wal/instance
I20260808 16:54:31.106539 30066 raft_consensus.cc:348] T 00000000000000000000000000000000 P 558963c9265449d693b9f291d37f44c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "558963c9265449d693b9f291d37f44c9" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 44301 } }
I20260808 16:54:31.106685 30066 raft_consensus.cc:374] T 00000000000000000000000000000000 P 558963c9265449d693b9f291d37f44c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:54:31.106714 8288 external_mini_cluster.cc:1428] Setting key 6856f5efdbe7b301a92ae0727130ae9a
I20260808 16:54:31.106710 30066 raft_consensus.cc:729] T 00000000000000000000000000000000 P 558963c9265449d693b9f291d37f44c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 558963c9265449d693b9f291d37f44c9, State: Initialized, Role: FOLLOWER
I20260808 16:54:31.106837 30066 consensus_queue.cc:260] T 00000000000000000000000000000000 P 558963c9265449d693b9f291d37f44c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "558963c9265449d693b9f291d37f44c9" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 44301 } }
I20260808 16:54:31.106912 30066 raft_consensus.cc:388] T 00000000000000000000000000000000 P 558963c9265449d693b9f291d37f44c9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 16:54:31.106932 30066 raft_consensus.cc:482] T 00000000000000000000000000000000 P 558963c9265449d693b9f291d37f44c9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 16:54:31.106976 30066 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 558963c9265449d693b9f291d37f44c9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:54:31.107800 30066 raft_consensus.cc:504] T 00000000000000000000000000000000 P 558963c9265449d693b9f291d37f44c9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "558963c9265449d693b9f291d37f44c9" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 44301 } }
I20260808 16:54:31.107887 30066 leader_election.cc:302] T 00000000000000000000000000000000 P 558963c9265449d693b9f291d37f44c9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 558963c9265449d693b9f291d37f44c9; no voters:
I20260808 16:54:31.108032 30066 leader_election.cc:288] T 00000000000000000000000000000000 P 558963c9265449d693b9f291d37f44c9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 16:54:31.108076 30074 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 558963c9265449d693b9f291d37f44c9 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:54:31.108217 30074 raft_consensus.cc:686] T 00000000000000000000000000000000 P 558963c9265449d693b9f291d37f44c9 [term 1 LEADER]: Becoming Leader. State: Replica: 558963c9265449d693b9f291d37f44c9, State: Running, Role: LEADER
I20260808 16:54:31.108274 30066 sys_catalog.cc:564] T 00000000000000000000000000000000 P 558963c9265449d693b9f291d37f44c9 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 16:54:31.108305 30074 consensus_queue.cc:237] T 00000000000000000000000000000000 P 558963c9265449d693b9f291d37f44c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "558963c9265449d693b9f291d37f44c9" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 44301 } }
I20260808 16:54:31.108696 30074 sys_catalog.cc:455] T 00000000000000000000000000000000 P 558963c9265449d693b9f291d37f44c9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 558963c9265449d693b9f291d37f44c9. Latest consensus state: current_term: 1 leader_uuid: "558963c9265449d693b9f291d37f44c9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "558963c9265449d693b9f291d37f44c9" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 44301 } } }
I20260808 16:54:31.108769 30074 sys_catalog.cc:458] T 00000000000000000000000000000000 P 558963c9265449d693b9f291d37f44c9 [sys.catalog]: This master's current role is: LEADER
I20260808 16:54:31.108997 30079 sys_catalog.cc:455] T 00000000000000000000000000000000 P 558963c9265449d693b9f291d37f44c9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "558963c9265449d693b9f291d37f44c9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "558963c9265449d693b9f291d37f44c9" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 44301 } } }
I20260808 16:54:31.109138 30079 sys_catalog.cc:458] T 00000000000000000000000000000000 P 558963c9265449d693b9f291d37f44c9 [sys.catalog]: This master's current role is: LEADER
I20260808 16:54:31.109454 30086 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 16:54:31.109918 30086 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 16:54:31.111887 30086 catalog_manager.cc:1347] Generated new cluster ID: 500a46cec5b8466893a955131566b59e
I20260808 16:54:31.111936 30086 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 16:54:31 dist-test-slave-6zbq krb5kdc[29921](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208070, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260808 16:54:31.117686 30072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:31.107290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53830 (local address 127.8.24.62:44301)
0808 16:54:31.107489 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0808 16:54:31.107496 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 16:54:31.107520 (+ 24us) server_negotiation.cc:407] Connection header received
0808 16:54:31.107601 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:54:31.107606 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:54:31.107652 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:54:31.107759 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:54:31.108514 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:31.109873 (+ 1359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:31.110928 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:31.111356 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:31.111433 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:31.113999 (+ 2566us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:54:31.114003 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:54:31.114003 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:54:31.114014 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:54:31.116227 (+ 2213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:54:31.116593 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:54:31.116595 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:54:31.116596 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:54:31.116698 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:54:31.116886 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:54:31.116886 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:54:31.116887 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:54:31.117142 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:54:31.117274 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0808 16:54:31.117377 (+ 103us) server_negotiation.cc:299] Negotiation successful
0808 16:54:31.117494 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":123,"threads_started":1}
I20260808 16:54:31.118588 8288 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 16:54:31.133757 30086 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 16:54:31.134199 30086 catalog_manager.cc:1504] Loading token signing keys...
I20260808 16:54:31.147172 30086 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 558963c9265449d693b9f291d37f44c9: Generated new TSK 0
I20260808 16:54:31.147320 30086 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 16:54:31.172189 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 29946
2026-08-08T16:54:31Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1401 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 16:54:31 dist-test-slave-6zbq krb5kdc[30116](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 16:54:31 dist-test-slave-6zbq krb5kdc[30116](info): set up 2 sockets
Aug 08 16:54:31 dist-test-slave-6zbq krb5kdc[30116](info): commencing operation
krb5kdc: starting...
W20260808 16:54:32.299778 8288 mini_kdc.cc:121] Time spent starting KDC: real 1.104s user 0.005s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/krb5kdc/test-user.keytab.
Aug 08 16:54:32 dist-test-slave-6zbq krb5kdc[30116](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208072, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T16:54:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T16:54:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.24.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
I20260808 16:54:32.406688 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:35859
--webserver_interface=127.8.24.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:46527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:35859
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:54:32.484215 30132 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:54:32.484388 30132 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:54:32.484408 30132 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:54:32.485446 30132 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 16:54:32.485478 30132 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:54:32.485491 30132 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:54:32.485502 30132 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 16:54:32.485515 30132 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 16:54:32.486886 30132 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:46527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:35859
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:35859
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.24.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30132
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:54:32.487057 30132 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:54:32.487222 30132 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:54:32.491626 30141 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:54:32.491706 30145 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:54:32.491760 30132 server_base.cc:1034] running on GCE node
W20260808 16:54:32.491627 30143 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:54:32.492105 30132 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:54:32.492367 30132 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:54:32.493537 30132 hybrid_clock.cc:648] HybridClock initialized: now 1786208072493524 us; error 29 us; skew 500 ppm
Aug 08 16:54:32 dist-test-slave-6zbq krb5kdc[30116](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208072, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:54:32.495930 30132 init.cc:377] Logged in from keytab as kudu/127.8.24.62@KRBTEST.COM (short username kudu)
I20260808 16:54:32.496342 30132 webserver.cc:460] Webserver started at http://127.8.24.62:42833/ using document root <none> and password file <none>
I20260808 16:54:32.496521 30132 fs_manager.cc:359] Metadata directory not provided
I20260808 16:54:32.496563 30132 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:54:32.496675 30132 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:54:32.497608 30132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/master-0/data/instance:
uuid: "18e9d95d961846578aecef36a0092b32"
format_stamp: "Formatted at 2026-08-08 16:54:32 on dist-test-slave-6zbq"
server_key: "b5b7b2e6fa7dfe87b7a3616d9f8fa617"
server_key_iv: "43423a4e4baf6068c563f9de2761e272"
server_key_version: "encryptionkey@0"
I20260808 16:54:32.497920 30132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/master-0/wal/instance:
uuid: "18e9d95d961846578aecef36a0092b32"
format_stamp: "Formatted at 2026-08-08 16:54:32 on dist-test-slave-6zbq"
server_key: "b5b7b2e6fa7dfe87b7a3616d9f8fa617"
server_key_iv: "43423a4e4baf6068c563f9de2761e272"
server_key_version: "encryptionkey@0"
I20260808 16:54:32.499123 30132 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 16:54:32.499725 30160 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:54:32.499854 30132 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 16:54:32.499936 30132 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/master-0/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/master-0/wal
uuid: "18e9d95d961846578aecef36a0092b32"
format_stamp: "Formatted at 2026-08-08 16:54:32 on dist-test-slave-6zbq"
server_key: "b5b7b2e6fa7dfe87b7a3616d9f8fa617"
server_key_iv: "43423a4e4baf6068c563f9de2761e272"
server_key_version: "encryptionkey@0"
I20260808 16:54:32.499995 30132 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:54:32.532131 30132 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:54:32.534044 30132 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:54:32.534169 30132 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:54:32.537742 30132 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.62:35859
I20260808 16:54:32.537771 30243 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.62:35859 every 8 connection(s)
I20260808 16:54:32.538142 30132 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
I20260808 16:54:32.539052 30245 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:54:32.540620 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 30132
I20260808 16:54:32.540733 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/master-0/wal/instance
I20260808 16:54:32.540922 8288 external_mini_cluster.cc:1428] Setting key 9f9d98ccd057d4ad9d894b47b5a58c3d
I20260808 16:54:32.541427 30245 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 18e9d95d961846578aecef36a0092b32: Bootstrap starting.
I20260808 16:54:32.542028 30245 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 18e9d95d961846578aecef36a0092b32: Neither blocks nor log segments found. Creating new log.
I20260808 16:54:32.542335 30245 log.cc:824] T 00000000000000000000000000000000 P 18e9d95d961846578aecef36a0092b32: Log is configured to *not* fsync() on all Append() calls
I20260808 16:54:32.543036 30245 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 18e9d95d961846578aecef36a0092b32: No bootstrap required, opened a new log
I20260808 16:54:32.545008 30245 raft_consensus.cc:348] T 00000000000000000000000000000000 P 18e9d95d961846578aecef36a0092b32 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18e9d95d961846578aecef36a0092b32" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35859 } }
I20260808 16:54:32.545135 30245 raft_consensus.cc:374] T 00000000000000000000000000000000 P 18e9d95d961846578aecef36a0092b32 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:54:32.545166 30245 raft_consensus.cc:729] T 00000000000000000000000000000000 P 18e9d95d961846578aecef36a0092b32 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18e9d95d961846578aecef36a0092b32, State: Initialized, Role: FOLLOWER
I20260808 16:54:32.545300 30245 consensus_queue.cc:260] T 00000000000000000000000000000000 P 18e9d95d961846578aecef36a0092b32 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18e9d95d961846578aecef36a0092b32" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35859 } }
I20260808 16:54:32.545403 30245 raft_consensus.cc:388] T 00000000000000000000000000000000 P 18e9d95d961846578aecef36a0092b32 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 16:54:32.545446 30245 raft_consensus.cc:482] T 00000000000000000000000000000000 P 18e9d95d961846578aecef36a0092b32 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 16:54:32.545497 30245 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 18e9d95d961846578aecef36a0092b32 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:54:32.546128 30245 raft_consensus.cc:504] T 00000000000000000000000000000000 P 18e9d95d961846578aecef36a0092b32 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18e9d95d961846578aecef36a0092b32" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35859 } }
I20260808 16:54:32.546244 30245 leader_election.cc:302] T 00000000000000000000000000000000 P 18e9d95d961846578aecef36a0092b32 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 18e9d95d961846578aecef36a0092b32; no voters:
I20260808 16:54:32.546427 30245 leader_election.cc:288] T 00000000000000000000000000000000 P 18e9d95d961846578aecef36a0092b32 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 16:54:32.546460 30253 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 18e9d95d961846578aecef36a0092b32 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:54:32.546629 30253 raft_consensus.cc:686] T 00000000000000000000000000000000 P 18e9d95d961846578aecef36a0092b32 [term 1 LEADER]: Becoming Leader. State: Replica: 18e9d95d961846578aecef36a0092b32, State: Running, Role: LEADER
I20260808 16:54:32.546674 30245 sys_catalog.cc:564] T 00000000000000000000000000000000 P 18e9d95d961846578aecef36a0092b32 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 16:54:32.546727 30253 consensus_queue.cc:237] T 00000000000000000000000000000000 P 18e9d95d961846578aecef36a0092b32 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18e9d95d961846578aecef36a0092b32" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35859 } }
I20260808 16:54:32.547029 30253 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18e9d95d961846578aecef36a0092b32 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 18e9d95d961846578aecef36a0092b32. Latest consensus state: current_term: 1 leader_uuid: "18e9d95d961846578aecef36a0092b32" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18e9d95d961846578aecef36a0092b32" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35859 } } }
I20260808 16:54:32.547083 30254 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18e9d95d961846578aecef36a0092b32 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "18e9d95d961846578aecef36a0092b32" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18e9d95d961846578aecef36a0092b32" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35859 } } }
I20260808 16:54:32.547227 30254 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18e9d95d961846578aecef36a0092b32 [sys.catalog]: This master's current role is: LEADER
Aug 08 16:54:32 dist-test-slave-6zbq krb5kdc[30116](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208072, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260808 16:54:32.547099 30253 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18e9d95d961846578aecef36a0092b32 [sys.catalog]: This master's current role is: LEADER
I20260808 16:54:32.547441 30269 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 16:54:32.547940 30269 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 16:54:32.549333 30269 catalog_manager.cc:1347] Generated new cluster ID: a0523afa43d445e2afc668410927b486
I20260808 16:54:32.549953 30269 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 16:54:32.551283 30251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:32.541390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57868 (local address 127.8.24.62:35859)
0808 16:54:32.541539 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0808 16:54:32.541547 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 16:54:32.541570 (+ 23us) server_negotiation.cc:407] Connection header received
0808 16:54:32.541992 (+ 422us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:54:32.541999 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:54:32.542041 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:54:32.542159 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:54:32.542876 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:32.543743 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:32.544812 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:32.545174 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:32.545268 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:32.547981 (+ 2713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:54:32.547985 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:54:32.547986 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:54:32.547997 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:54:32.549801 (+ 1804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:54:32.550128 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:54:32.550131 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:54:32.550131 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:54:32.550219 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:54:32.550433 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:54:32.550434 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:54:32.550434 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:54:32.550674 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:54:32.550766 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 16:54:32.551006 (+ 240us) server_negotiation.cc:299] Negotiation successful
0808 16:54:32.551133 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":61,"threads_started":1}
I20260808 16:54:32.559432 30269 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 16:54:32.560026 30269 catalog_manager.cc:1504] Loading token signing keys...
I20260808 16:54:32.565368 30269 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 18e9d95d961846578aecef36a0092b32: Generated new TSK 0
I20260808 16:54:32.565516 30269 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.24.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.1.keytab.
Entry for principal kudu/127.8.24.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.1.keytab.
I20260808 16:54:32.598173 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.1:0
--local_ip_for_outbound_sockets=127.8.24.1
--webserver_interface=127.8.24.1
--webserver_port=0
--tserver_master_addrs=127.8.24.62:35859
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.1.keytab
--principal=kudu/127.8.24.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:46527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:54:32.679021 30299 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:54:32.679200 30299 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:54:32.679222 30299 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:54:32.680275 30299 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:54:32.680306 30299 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:54:32.680330 30299 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.1
I20260808 16:54:32.681668 30299 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:46527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.1.keytab
--principal=kudu/127.8.24.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.24.1
--webserver_port=0
--tserver_master_addrs=127.8.24.62:35859
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30299
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.1
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:54:32.681864 30299 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:54:32.682062 30299 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:54:32.686600 30316 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:54:32.686607 30299 server_base.cc:1034] running on GCE node
W20260808 16:54:32.686575 30317 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:54:32.686563 30319 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:54:32.686973 30299 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:54:32.687170 30299 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:54:32.688313 30299 hybrid_clock.cc:648] HybridClock initialized: now 1786208072688296 us; error 27 us; skew 500 ppm
Aug 08 16:54:32 dist-test-slave-6zbq krb5kdc[30116](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208072, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:54:32.690543 30299 init.cc:377] Logged in from keytab as kudu/127.8.24.1@KRBTEST.COM (short username kudu)
I20260808 16:54:32.690912 30299 webserver.cc:460] Webserver started at http://127.8.24.1:33289/ using document root <none> and password file <none>
I20260808 16:54:32.691092 30299 fs_manager.cc:359] Metadata directory not provided
I20260808 16:54:32.691135 30299 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:54:32.691264 30299 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:54:32.692221 30299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-0/data/instance:
uuid: "e2efec6d440c4e2391890da0bd7e3a92"
format_stamp: "Formatted at 2026-08-08 16:54:32 on dist-test-slave-6zbq"
server_key: "1c6cf7513357174cb09007eca6f374ec"
server_key_iv: "ae6589c1f3ec02c93e70e6a5758f59d9"
server_key_version: "encryptionkey@0"
I20260808 16:54:32.692528 30299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-0/wal/instance:
uuid: "e2efec6d440c4e2391890da0bd7e3a92"
format_stamp: "Formatted at 2026-08-08 16:54:32 on dist-test-slave-6zbq"
server_key: "1c6cf7513357174cb09007eca6f374ec"
server_key_iv: "ae6589c1f3ec02c93e70e6a5758f59d9"
server_key_version: "encryptionkey@0"
I20260808 16:54:32.693683 30299 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 16:54:32.694283 30329 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:54:32.694427 30299 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 16:54:32.694518 30299 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-0/wal
uuid: "e2efec6d440c4e2391890da0bd7e3a92"
format_stamp: "Formatted at 2026-08-08 16:54:32 on dist-test-slave-6zbq"
server_key: "1c6cf7513357174cb09007eca6f374ec"
server_key_iv: "ae6589c1f3ec02c93e70e6a5758f59d9"
server_key_version: "encryptionkey@0"
I20260808 16:54:32.694577 30299 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:54:32.703039 30299 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:54:32.704913 30299 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:54:32.705034 30299 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:54:32.705207 30299 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:54:32.705459 30299 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:54:32.705492 30299 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:54:32.705511 30299 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:54:32.705533 30299 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:54:32.710824 30299 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.1:39989
I20260808 16:54:32.710888 30507 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.1:39989 every 8 connection(s)
I20260808 16:54:32.711120 30299 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-0/data/info.pb
I20260808 16:54:32.712248 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 30299
I20260808 16:54:32.712344 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-0/wal/instance
I20260808 16:54:32.712519 8288 external_mini_cluster.cc:1428] Setting key 3646dd7b197d3d669aba2dc68cd95ec6
Aug 08 16:54:32 dist-test-slave-6zbq krb5kdc[30116](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208072, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.1@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260808 16:54:32.719729 30251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:32.711711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:46275 (local address 127.8.24.62:35859)
0808 16:54:32.711773 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 16:54:32.711775 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:54:32.711927 (+ 152us) server_negotiation.cc:407] Connection header received
0808 16:54:32.712353 (+ 426us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:54:32.712354 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:54:32.712366 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:54:32.712414 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:54:32.713311 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:32.713804 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:32.714946 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:32.715149 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:32.715203 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:32.717305 (+ 2102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:54:32.717307 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:54:32.717307 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:54:32.717315 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:54:32.718526 (+ 1211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:54:32.718883 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:54:32.718884 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:54:32.718885 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:54:32.718938 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:54:32.719145 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:54:32.719146 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:54:32.719147 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:54:32.719260 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:54:32.719314 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 16:54:32.719555 (+ 241us) server_negotiation.cc:299] Negotiation successful
0808 16:54:32.719642 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 16:54:32.719873 30510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:32.711672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:35859 (local address 127.8.24.1:46275)
0808 16:54:32.711820 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:32.711856 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 16:54:32.712261 (+ 405us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:54:32.712518 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:54:32.712521 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:54:32.713044 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:54:32.713219 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:54:32.713225 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:54:32.713884 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:54:32.713886 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:54:32.714865 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:54:32.714868 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:54:32.715223 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:54:32.715224 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:54:32.715352 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:32.715781 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:54:32.715789 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:54:32.717221 (+ 1432us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:54:32.718604 (+ 1383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:54:32.718610 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:54:32.718612 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:54:32.718818 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:54:32.718990 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:54:32.718991 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:54:32.718992 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:54:32.719091 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:54:32.719314 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:54:32.719321 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:54:32.719444 (+ 123us) client_negotiation.cc:769] Sending connection context
0808 16:54:32.719527 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 16:54:32.719668 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":86,"threads_started":1}
I20260808 16:54:32.720322 30508 heartbeater.cc:347] Connected to a master server at 127.8.24.62:35859
I20260808 16:54:32.720417 30508 heartbeater.cc:464] Registering TS with master...
I20260808 16:54:32.720582 30508 heartbeater.cc:511] Master 127.8.24.62:35859 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.8.24.2@KRBTEST.COM; defaulting to no policy
I20260808 16:54:32.721065 30192 ts_manager.cc:194] Registered new tserver with Master: e2efec6d440c4e2391890da0bd7e3a92 (127.8.24.1:39989)
I20260808 16:54:32.721756 30192 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.1@KRBTEST.COM'} at 127.8.24.1:46275
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.2.keytab.
Entry for principal kudu/127.8.24.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.2.keytab.
I20260808 16:54:32.734756 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.2:0
--local_ip_for_outbound_sockets=127.8.24.2
--webserver_interface=127.8.24.2
--webserver_port=0
--tserver_master_addrs=127.8.24.62:35859
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.2.keytab
--principal=kudu/127.8.24.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:46527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:54:32.812672 30516 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:54:32.812850 30516 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:54:32.812880 30516 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:54:32.813951 30516 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:54:32.813988 30516 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:54:32.814023 30516 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.2
I20260808 16:54:32.815409 30516 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:46527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.2.keytab
--principal=kudu/127.8.24.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.24.2
--webserver_port=0
--tserver_master_addrs=127.8.24.62:35859
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30516
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.2
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:54:32.815634 30516 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:54:32.815825 30516 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:54:32.820329 30524 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:54:32.820310 30526 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:54:32.820298 30532 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:54:32.820386 30516 server_base.cc:1034] running on GCE node
I20260808 16:54:32.820731 30516 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:54:32.820945 30516 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:54:32.822073 30516 hybrid_clock.cc:648] HybridClock initialized: now 1786208072822060 us; error 31 us; skew 500 ppm
Aug 08 16:54:32 dist-test-slave-6zbq krb5kdc[30116](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208072, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:54:32.824309 30516 init.cc:377] Logged in from keytab as kudu/127.8.24.2@KRBTEST.COM (short username kudu)
I20260808 16:54:32.824708 30516 webserver.cc:460] Webserver started at http://127.8.24.2:45533/ using document root <none> and password file <none>
I20260808 16:54:32.824882 30516 fs_manager.cc:359] Metadata directory not provided
I20260808 16:54:32.824923 30516 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:54:32.825033 30516 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:54:32.825938 30516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-1/data/instance:
uuid: "57fe315458094d32bff19b928d7f3e28"
format_stamp: "Formatted at 2026-08-08 16:54:32 on dist-test-slave-6zbq"
server_key: "2778c3b10f02cc8146e641346f5e74ab"
server_key_iv: "1a2dd2c36087f0ba9f98d3afcd71d2bf"
server_key_version: "encryptionkey@0"
I20260808 16:54:32.826237 30516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-1/wal/instance:
uuid: "57fe315458094d32bff19b928d7f3e28"
format_stamp: "Formatted at 2026-08-08 16:54:32 on dist-test-slave-6zbq"
server_key: "2778c3b10f02cc8146e641346f5e74ab"
server_key_iv: "1a2dd2c36087f0ba9f98d3afcd71d2bf"
server_key_version: "encryptionkey@0"
I20260808 16:54:32.827394 30516 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 16:54:32.828015 30544 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:54:32.828158 30516 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 16:54:32.828230 30516 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-1/wal
uuid: "57fe315458094d32bff19b928d7f3e28"
format_stamp: "Formatted at 2026-08-08 16:54:32 on dist-test-slave-6zbq"
server_key: "2778c3b10f02cc8146e641346f5e74ab"
server_key_iv: "1a2dd2c36087f0ba9f98d3afcd71d2bf"
server_key_version: "encryptionkey@0"
I20260808 16:54:32.828295 30516 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:54:32.844445 30516 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:54:32.846035 30516 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:54:32.846133 30516 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:54:32.846328 30516 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:54:32.846689 30516 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:54:32.846724 30516 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:54:32.846742 30516 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:54:32.846764 30516 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:54:32.852389 30516 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.2:37085
I20260808 16:54:32.852427 30738 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.2:37085 every 8 connection(s)
I20260808 16:54:32.852679 30516 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-1/data/info.pb
Aug 08 16:54:32 dist-test-slave-6zbq krb5kdc[30116](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208072, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.2@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260808 16:54:32.858744 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 30516
I20260808 16:54:32.858834 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-1/wal/instance
I20260808 16:54:32.858997 8288 external_mini_cluster.cc:1428] Setting key 0d52e99b2528e6ab6ccc6b1e45745e81
I20260808 16:54:32.860703 30251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:32.853275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:58985 (local address 127.8.24.62:35859)
0808 16:54:32.853323 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 16:54:32.853325 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:54:32.853490 (+ 165us) server_negotiation.cc:407] Connection header received
0808 16:54:32.853910 (+ 420us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:54:32.853911 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:54:32.853922 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:54:32.853971 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:54:32.854797 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:32.855290 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:32.856154 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:32.856333 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:32.856380 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:32.858092 (+ 1712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:54:32.858093 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:54:32.858093 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:54:32.858100 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:54:32.859478 (+ 1378us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:54:32.859900 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:54:32.859901 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:54:32.859901 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:54:32.859953 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:54:32.860180 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:54:32.860181 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:54:32.860181 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:54:32.860281 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:54:32.860325 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0808 16:54:32.860588 (+ 263us) server_negotiation.cc:299] Negotiation successful
0808 16:54:32.860644 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 16:54:32.860904 30743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:32.853249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:35859 (local address 127.8.24.2:58985)
0808 16:54:32.853386 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:32.853417 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0808 16:54:32.853811 (+ 394us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:54:32.854018 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:54:32.854021 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:54:32.854549 (+ 528us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:54:32.854723 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:54:32.854727 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:54:32.855354 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:54:32.855354 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:54:32.856086 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:54:32.856088 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:54:32.856388 (+ 300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:54:32.856388 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:54:32.856471 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:32.856812 (+ 341us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:54:32.856819 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:54:32.858012 (+ 1193us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:54:32.859548 (+ 1536us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:54:32.859555 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:54:32.859557 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:54:32.859834 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:54:32.860008 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:54:32.860009 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:54:32.860009 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:54:32.860127 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:54:32.860330 (+ 203us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:54:32.860334 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:54:32.860477 (+ 143us) client_negotiation.cc:769] Sending connection context
0808 16:54:32.860576 (+ 99us) client_negotiation.cc:240] Negotiation successful
0808 16:54:32.860720 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":80,"threads_started":1}
I20260808 16:54:32.861334 30739 heartbeater.cc:347] Connected to a master server at 127.8.24.62:35859
I20260808 16:54:32.861429 30739 heartbeater.cc:464] Registering TS with master...
I20260808 16:54:32.861585 30739 heartbeater.cc:511] Master 127.8.24.62:35859 requested a full tablet report, sending...
I20260808 16:54:32.861959 30192 ts_manager.cc:194] Registered new tserver with Master: 57fe315458094d32bff19b928d7f3e28 (127.8.24.2:37085)
I20260808 16:54:32.862440 30192 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.2@KRBTEST.COM'} at 127.8.24.2:58985
WARNING: no policy specified for kudu/127.8.24.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.3.keytab.
Entry for principal kudu/127.8.24.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.3.keytab.
I20260808 16:54:32.880316 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.3:0
--local_ip_for_outbound_sockets=127.8.24.3
--webserver_interface=127.8.24.3
--webserver_port=0
--tserver_master_addrs=127.8.24.62:35859
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.3.keytab
--principal=kudu/127.8.24.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:46527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:54:32.958261 30749 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:54:32.958453 30749 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:54:32.958477 30749 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:54:32.959554 30749 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:54:32.959594 30749 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:54:32.959621 30749 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.3
I20260808 16:54:32.961006 30749 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:46527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.3.keytab
--principal=kudu/127.8.24.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.24.3
--webserver_port=0
--tserver_master_addrs=127.8.24.62:35859
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30749
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.3
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:54:32.961216 30749 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:54:32.961421 30749 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:54:32.965844 30755 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:54:32.965869 30760 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:54:32.965883 30758 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:54:32.966028 30749 server_base.cc:1034] running on GCE node
I20260808 16:54:32.966218 30749 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:54:32.966464 30749 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:54:32.967592 30749 hybrid_clock.cc:648] HybridClock initialized: now 1786208072967582 us; error 28 us; skew 500 ppm
Aug 08 16:54:32 dist-test-slave-6zbq krb5kdc[30116](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208072, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:54:32.969830 30749 init.cc:377] Logged in from keytab as kudu/127.8.24.3@KRBTEST.COM (short username kudu)
I20260808 16:54:32.970232 30749 webserver.cc:460] Webserver started at http://127.8.24.3:44891/ using document root <none> and password file <none>
I20260808 16:54:32.970490 30749 fs_manager.cc:359] Metadata directory not provided
I20260808 16:54:32.970541 30749 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:54:32.970662 30749 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:54:32.971606 30749 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-2/data/instance:
uuid: "396866376cb04432abf9245607b453c2"
format_stamp: "Formatted at 2026-08-08 16:54:32 on dist-test-slave-6zbq"
server_key: "3345e3d822c4b152c4f06343904934b9"
server_key_iv: "ad80ce4eda7303fb9501cc055e140da7"
server_key_version: "encryptionkey@0"
I20260808 16:54:32.971922 30749 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-2/wal/instance:
uuid: "396866376cb04432abf9245607b453c2"
format_stamp: "Formatted at 2026-08-08 16:54:32 on dist-test-slave-6zbq"
server_key: "3345e3d822c4b152c4f06343904934b9"
server_key_iv: "ad80ce4eda7303fb9501cc055e140da7"
server_key_version: "encryptionkey@0"
I20260808 16:54:32.973047 30749 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 16:54:32.973664 30774 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:54:32.973829 30749 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 16:54:32.973917 30749 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-2/wal
uuid: "396866376cb04432abf9245607b453c2"
format_stamp: "Formatted at 2026-08-08 16:54:32 on dist-test-slave-6zbq"
server_key: "3345e3d822c4b152c4f06343904934b9"
server_key_iv: "ad80ce4eda7303fb9501cc055e140da7"
server_key_version: "encryptionkey@0"
I20260808 16:54:32.973980 30749 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:54:32.981704 30749 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:54:32.983271 30749 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:54:32.983385 30749 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:54:32.983570 30749 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:54:32.983822 30749 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:54:32.983853 30749 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:54:32.983878 30749 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:54:32.983902 30749 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:54:32.989084 30749 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.3:37411
I20260808 16:54:32.989142 30958 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.3:37411 every 8 connection(s)
I20260808 16:54:32.989389 30749 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-2/data/info.pb
I20260808 16:54:32.994194 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 30749
I20260808 16:54:32.994303 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-2/wal/instance
I20260808 16:54:32.994515 8288 external_mini_cluster.cc:1428] Setting key 196fc9f208ee9b78eeda4969ba631e93
Aug 08 16:54:32 dist-test-slave-6zbq krb5kdc[30116](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208072, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.3@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260808 16:54:32.997149 30251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:32.989959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:58411 (local address 127.8.24.62:35859)
0808 16:54:32.990004 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 16:54:32.990006 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:54:32.990171 (+ 165us) server_negotiation.cc:407] Connection header received
0808 16:54:32.990664 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:54:32.990665 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:54:32.990675 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:54:32.990713 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:54:32.991508 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:32.991987 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:32.992860 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:32.993034 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:32.993075 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:32.994990 (+ 1915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:54:32.994992 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:54:32.994992 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:54:32.994999 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:54:32.996146 (+ 1147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:54:32.996464 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:54:32.996465 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:54:32.996465 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:54:32.996507 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:54:32.996684 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:54:32.996684 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:54:32.996684 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:54:32.996782 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:54:32.996829 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0808 16:54:32.997008 (+ 179us) server_negotiation.cc:299] Negotiation successful
0808 16:54:32.997063 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 16:54:32.997323 30964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:32.989923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:35859 (local address 127.8.24.3:58411)
0808 16:54:32.990067 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:32.990104 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 16:54:32.990565 (+ 461us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:54:32.990769 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:54:32.990772 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:54:32.991269 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:54:32.991445 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:54:32.991449 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:54:32.992045 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:54:32.992045 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:54:32.992802 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:54:32.992804 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:54:32.993081 (+ 277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:54:32.993082 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:54:32.993168 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:32.993515 (+ 347us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:54:32.993522 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:54:32.994906 (+ 1384us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:54:32.996208 (+ 1302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:54:32.996214 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:54:32.996215 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:54:32.996410 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:54:32.996548 (+ 138us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:54:32.996549 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:54:32.996549 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:54:32.996640 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:54:32.996825 (+ 185us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:54:32.996827 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:54:32.996922 (+ 95us) client_negotiation.cc:769] Sending connection context
0808 16:54:32.996991 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 16:54:32.997139 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":67,"threads_started":1}
I20260808 16:54:32.997720 30960 heartbeater.cc:347] Connected to a master server at 127.8.24.62:35859
I20260808 16:54:32.997804 30960 heartbeater.cc:464] Registering TS with master...
I20260808 16:54:32.997937 30960 heartbeater.cc:511] Master 127.8.24.62:35859 requested a full tablet report, sending...
I20260808 16:54:32.998260 30192 ts_manager.cc:194] Registered new tserver with Master: 396866376cb04432abf9245607b453c2 (127.8.24.3:37411)
I20260808 16:54:32.998617 30192 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.3@KRBTEST.COM'} at 127.8.24.3:58411
I20260808 16:54:33.006000 8288 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 16:54:33.011973 30251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:33.007233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57874 (local address 127.8.24.62:35859)
0808 16:54:33.007281 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 16:54:33.007282 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 16:54:33.007390 (+ 108us) server_negotiation.cc:407] Connection header received
0808 16:54:33.007427 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:54:33.007428 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:54:33.007436 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:54:33.007477 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:54:33.007949 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:33.008383 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:33.009011 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:33.009170 (+ 159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:33.009206 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:33.010152 (+ 946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:54:33.010153 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:54:33.010153 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:54:33.010157 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:54:33.011202 (+ 1045us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:54:33.011470 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:54:33.011470 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:54:33.011471 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:54:33.011511 (+ 40us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:54:33.011618 (+ 107us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:54:33.011619 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:54:33.011619 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:54:33.011725 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:54:33.011793 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 16:54:33.011867 (+ 74us) server_negotiation.cc:299] Negotiation successful
0808 16:54:33.011920 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 16:54:33.013314 30192 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57874:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 16:54:33.014889 30192 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 16:54:33.020452 30977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:33.016751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.2:37085 (local address 127.0.0.1:51466)
0808 16:54:33.016983 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:33.017026 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 16:54:33.017090 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:54:33.017304 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:54:33.017306 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:54:33.017308 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:54:33.017362 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:54:33.017363 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:54:33.018311 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:54:33.018312 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:54:33.019354 (+ 1042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:54:33.019357 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:54:33.020182 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:54:33.020183 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:54:33.020266 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:33.020277 (+ 11us) client_negotiation.cc:769] Sending connection context
0808 16:54:33.020333 (+ 56us) client_negotiation.cc:240] Negotiation successful
0808 16:54:33.020382 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":215,"mutex_wait_us":2,"thread_start_us":68,"threads_started":1}
I20260808 16:54:33.020470 30981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:33.016974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51466 (local address 127.8.24.2:37085)
0808 16:54:33.017134 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0808 16:54:33.017138 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 16:54:33.017148 (+ 10us) server_negotiation.cc:407] Connection header received
0808 16:54:33.017185 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:54:33.017187 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:54:33.017203 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:54:33.017306 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:54:33.017449 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:33.018218 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:33.019449 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:33.020114 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:33.020172 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:33.020226 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 16:54:33.020355 (+ 129us) server_negotiation.cc:299] Negotiation successful
0808 16:54:33.020393 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":97,"threads_started":1}
I20260808 16:54:33.020918 30976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:33.016764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.3:37411 (local address 127.0.0.1:33240)
0808 16:54:33.017282 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:33.017292 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 16:54:33.017346 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:54:33.017542 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:54:33.017543 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:54:33.017545 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:54:33.017580 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:54:33.017581 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:54:33.018815 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:54:33.018816 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:54:33.019876 (+ 1060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:54:33.019879 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:54:33.020703 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:54:33.020705 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:54:33.020765 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:33.020769 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 16:54:33.020818 (+ 49us) client_negotiation.cc:240] Negotiation successful
0808 16:54:33.020854 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":505,"mutex_wait_us":31,"thread_start_us":43,"threads_started":1}
I20260808 16:54:33.020944 30979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:33.016941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33240 (local address 127.8.24.3:37411)
0808 16:54:33.017210 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0808 16:54:33.017214 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 16:54:33.017354 (+ 140us) server_negotiation.cc:407] Connection header received
0808 16:54:33.017394 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:54:33.017396 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:54:33.017411 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:54:33.017478 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:54:33.017663 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:33.018711 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:33.019940 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:33.020628 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:33.020696 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:33.020745 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 16:54:33.020841 (+ 96us) server_negotiation.cc:299] Negotiation successful
0808 16:54:33.020876 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":246,"thread_start_us":54,"threads_started":1}
I20260808 16:54:33.021112 30635 tablet_service.cc:1467] Processing CreateTablet for tablet bb32b7eee99149a680244b0413545ab1 (DEFAULT_TABLE table=test-table [id=b382a25050b94de081af3e577a5f90fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:54:33.021377 30978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:33.016751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:39989 (local address 127.0.0.1:56872)
0808 16:54:33.017030 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:33.017038 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 16:54:33.017090 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:54:33.017674 (+ 584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:54:33.017675 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:54:33.017678 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:54:33.017718 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:54:33.017720 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:54:33.018969 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:54:33.018970 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:54:33.019818 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:54:33.019821 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:54:33.021185 (+ 1364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:54:33.021186 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:54:33.021236 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:33.021239 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 16:54:33.021272 (+ 33us) client_negotiation.cc:240] Negotiation successful
0808 16:54:33.021310 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":85,"threads_started":1}
I20260808 16:54:33.021407 30635 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bb32b7eee99149a680244b0413545ab1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:54:33.021680 30980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:33.016943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56872 (local address 127.8.24.1:39989)
0808 16:54:33.017461 (+ 518us) server_negotiation.cc:206] Beginning negotiation
0808 16:54:33.017465 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 16:54:33.017474 (+ 9us) server_negotiation.cc:407] Connection header received
0808 16:54:33.017511 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:54:33.017513 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:54:33.017527 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:54:33.017595 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:54:33.017820 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:33.018885 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:33.019889 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:33.020900 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:33.021453 (+ 553us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:33.021528 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 16:54:33.021553 (+ 25us) server_negotiation.cc:299] Negotiation successful
0808 16:54:33.021610 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":479,"thread_start_us":71,"threads_started":1}
I20260808 16:54:33.022212 30394 tablet_service.cc:1467] Processing CreateTablet for tablet bb32b7eee99149a680244b0413545ab1 (DEFAULT_TABLE table=test-table [id=b382a25050b94de081af3e577a5f90fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:54:33.022513 30394 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bb32b7eee99149a680244b0413545ab1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:54:33.022953 30853 tablet_service.cc:1467] Processing CreateTablet for tablet bb32b7eee99149a680244b0413545ab1 (DEFAULT_TABLE table=test-table [id=b382a25050b94de081af3e577a5f90fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:54:33.023253 30853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bb32b7eee99149a680244b0413545ab1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:54:33.024107 30982 tablet_bootstrap.cc:492] T bb32b7eee99149a680244b0413545ab1 P 57fe315458094d32bff19b928d7f3e28: Bootstrap starting.
I20260808 16:54:33.024775 30982 tablet_bootstrap.cc:654] T bb32b7eee99149a680244b0413545ab1 P 57fe315458094d32bff19b928d7f3e28: Neither blocks nor log segments found. Creating new log.
I20260808 16:54:33.025218 30982 log.cc:824] T bb32b7eee99149a680244b0413545ab1 P 57fe315458094d32bff19b928d7f3e28: Log is configured to *not* fsync() on all Append() calls
I20260808 16:54:33.025859 30982 tablet_bootstrap.cc:492] T bb32b7eee99149a680244b0413545ab1 P 57fe315458094d32bff19b928d7f3e28: No bootstrap required, opened a new log
I20260808 16:54:33.025919 30982 ts_tablet_manager.cc:1397] T bb32b7eee99149a680244b0413545ab1 P 57fe315458094d32bff19b928d7f3e28: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 16:54:33.026103 30988 tablet_bootstrap.cc:492] T bb32b7eee99149a680244b0413545ab1 P 396866376cb04432abf9245607b453c2: Bootstrap starting.
I20260808 16:54:33.026638 30988 tablet_bootstrap.cc:654] T bb32b7eee99149a680244b0413545ab1 P 396866376cb04432abf9245607b453c2: Neither blocks nor log segments found. Creating new log.
I20260808 16:54:33.027029 30988 log.cc:824] T bb32b7eee99149a680244b0413545ab1 P 396866376cb04432abf9245607b453c2: Log is configured to *not* fsync() on all Append() calls
I20260808 16:54:33.027385 30982 raft_consensus.cc:348] T bb32b7eee99149a680244b0413545ab1 P 57fe315458094d32bff19b928d7f3e28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57fe315458094d32bff19b928d7f3e28" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 37085 } } peers { permanent_uuid: "e2efec6d440c4e2391890da0bd7e3a92" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 39989 } } peers { permanent_uuid: "396866376cb04432abf9245607b453c2" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 37411 } }
I20260808 16:54:33.027513 30982 raft_consensus.cc:374] T bb32b7eee99149a680244b0413545ab1 P 57fe315458094d32bff19b928d7f3e28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:54:33.027534 30982 raft_consensus.cc:729] T bb32b7eee99149a680244b0413545ab1 P 57fe315458094d32bff19b928d7f3e28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57fe315458094d32bff19b928d7f3e28, State: Initialized, Role: FOLLOWER
I20260808 16:54:33.027657 30988 tablet_bootstrap.cc:492] T bb32b7eee99149a680244b0413545ab1 P 396866376cb04432abf9245607b453c2: No bootstrap required, opened a new log
I20260808 16:54:33.027642 30982 consensus_queue.cc:260] T bb32b7eee99149a680244b0413545ab1 P 57fe315458094d32bff19b928d7f3e28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57fe315458094d32bff19b928d7f3e28" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 37085 } } peers { permanent_uuid: "e2efec6d440c4e2391890da0bd7e3a92" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 39989 } } peers { permanent_uuid: "396866376cb04432abf9245607b453c2" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 37411 } }
I20260808 16:54:33.027724 30988 ts_tablet_manager.cc:1397] T bb32b7eee99149a680244b0413545ab1 P 396866376cb04432abf9245607b453c2: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 16:54:33.027882 30982 ts_tablet_manager.cc:1428] T bb32b7eee99149a680244b0413545ab1 P 57fe315458094d32bff19b928d7f3e28: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 16:54:33.027902 30739 heartbeater.cc:503] Master 127.8.24.62:35859 was elected leader, sending a full tablet report...
I20260808 16:54:33.027971 30986 tablet_bootstrap.cc:492] T bb32b7eee99149a680244b0413545ab1 P e2efec6d440c4e2391890da0bd7e3a92: Bootstrap starting.
I20260808 16:54:33.028535 30986 tablet_bootstrap.cc:654] T bb32b7eee99149a680244b0413545ab1 P e2efec6d440c4e2391890da0bd7e3a92: Neither blocks nor log segments found. Creating new log.
I20260808 16:54:33.029037 30986 log.cc:824] T bb32b7eee99149a680244b0413545ab1 P e2efec6d440c4e2391890da0bd7e3a92: Log is configured to *not* fsync() on all Append() calls
I20260808 16:54:33.029691 30986 tablet_bootstrap.cc:492] T bb32b7eee99149a680244b0413545ab1 P e2efec6d440c4e2391890da0bd7e3a92: No bootstrap required, opened a new log
I20260808 16:54:33.029743 30986 ts_tablet_manager.cc:1397] T bb32b7eee99149a680244b0413545ab1 P e2efec6d440c4e2391890da0bd7e3a92: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260808 16:54:33.029706 30988 raft_consensus.cc:348] T bb32b7eee99149a680244b0413545ab1 P 396866376cb04432abf9245607b453c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57fe315458094d32bff19b928d7f3e28" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 37085 } } peers { permanent_uuid: "e2efec6d440c4e2391890da0bd7e3a92" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 39989 } } peers { permanent_uuid: "396866376cb04432abf9245607b453c2" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 37411 } }
I20260808 16:54:33.029863 30988 raft_consensus.cc:374] T bb32b7eee99149a680244b0413545ab1 P 396866376cb04432abf9245607b453c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:54:33.029904 30988 raft_consensus.cc:729] T bb32b7eee99149a680244b0413545ab1 P 396866376cb04432abf9245607b453c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 396866376cb04432abf9245607b453c2, State: Initialized, Role: FOLLOWER
I20260808 16:54:33.030014 30988 consensus_queue.cc:260] T bb32b7eee99149a680244b0413545ab1 P 396866376cb04432abf9245607b453c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57fe315458094d32bff19b928d7f3e28" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 37085 } } peers { permanent_uuid: "e2efec6d440c4e2391890da0bd7e3a92" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 39989 } } peers { permanent_uuid: "396866376cb04432abf9245607b453c2" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 37411 } }
I20260808 16:54:33.030257 30988 ts_tablet_manager.cc:1428] T bb32b7eee99149a680244b0413545ab1 P 396866376cb04432abf9245607b453c2: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260808 16:54:33.030257 30960 heartbeater.cc:503] Master 127.8.24.62:35859 was elected leader, sending a full tablet report...
I20260808 16:54:33.031378 30986 raft_consensus.cc:348] T bb32b7eee99149a680244b0413545ab1 P e2efec6d440c4e2391890da0bd7e3a92 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57fe315458094d32bff19b928d7f3e28" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 37085 } } peers { permanent_uuid: "e2efec6d440c4e2391890da0bd7e3a92" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 39989 } } peers { permanent_uuid: "396866376cb04432abf9245607b453c2" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 37411 } }
I20260808 16:54:33.031512 30986 raft_consensus.cc:374] T bb32b7eee99149a680244b0413545ab1 P e2efec6d440c4e2391890da0bd7e3a92 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:54:33.031533 30986 raft_consensus.cc:729] T bb32b7eee99149a680244b0413545ab1 P e2efec6d440c4e2391890da0bd7e3a92 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2efec6d440c4e2391890da0bd7e3a92, State: Initialized, Role: FOLLOWER
I20260808 16:54:33.031612 30986 consensus_queue.cc:260] T bb32b7eee99149a680244b0413545ab1 P e2efec6d440c4e2391890da0bd7e3a92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57fe315458094d32bff19b928d7f3e28" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 37085 } } peers { permanent_uuid: "e2efec6d440c4e2391890da0bd7e3a92" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 39989 } } peers { permanent_uuid: "396866376cb04432abf9245607b453c2" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 37411 } }
I20260808 16:54:33.031853 30986 ts_tablet_manager.cc:1428] T bb32b7eee99149a680244b0413545ab1 P e2efec6d440c4e2391890da0bd7e3a92: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 16:54:33.031890 30508 heartbeater.cc:503] Master 127.8.24.62:35859 was elected leader, sending a full tablet report...
I20260808 16:54:33.050844 30998 raft_consensus.cc:482] T bb32b7eee99149a680244b0413545ab1 P e2efec6d440c4e2391890da0bd7e3a92 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 16:54:33.050925 30998 raft_consensus.cc:504] T bb32b7eee99149a680244b0413545ab1 P e2efec6d440c4e2391890da0bd7e3a92 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57fe315458094d32bff19b928d7f3e28" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 37085 } } peers { permanent_uuid: "e2efec6d440c4e2391890da0bd7e3a92" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 39989 } } peers { permanent_uuid: "396866376cb04432abf9245607b453c2" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 37411 } }
I20260808 16:54:33.051196 30998 leader_election.cc:288] T bb32b7eee99149a680244b0413545ab1 P e2efec6d440c4e2391890da0bd7e3a92 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57fe315458094d32bff19b928d7f3e28 (127.8.24.2:37085), 396866376cb04432abf9245607b453c2 (127.8.24.3:37411)
I20260808 16:54:33.054569 30981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:33.051391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:42439 (local address 127.8.24.2:37085)
0808 16:54:33.051436 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 16:54:33.051438 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:54:33.051524 (+ 86us) server_negotiation.cc:407] Connection header received
0808 16:54:33.051547 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:54:33.051548 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:54:33.051558 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:54:33.051598 (+ 40us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:54:33.051736 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:33.052412 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:33.053682 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:33.054241 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:33.054299 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:33.054307 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 16:54:33.054476 (+ 169us) server_negotiation.cc:299] Negotiation successful
0808 16:54:33.054509 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 16:54:33.054569 30979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:33.051470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:33713 (local address 127.8.24.3:37411)
0808 16:54:33.051501 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0808 16:54:33.051502 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 16:54:33.051622 (+ 120us) server_negotiation.cc:407] Connection header received
0808 16:54:33.051650 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:54:33.051651 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:54:33.051658 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:54:33.051694 (+ 36us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:54:33.051830 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:33.052482 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:33.053680 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:33.054225 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:33.054299 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:33.054307 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 16:54:33.054475 (+ 168us) server_negotiation.cc:299] Negotiation successful
0808 16:54:33.054509 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 16:54:33.054605 30510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:33.051329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.2:37085 (local address 127.8.24.1:42439)
0808 16:54:33.051418 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:33.051428 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 16:54:33.051476 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:54:33.051618 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:54:33.051619 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:54:33.051620 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:54:33.051677 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:54:33.051678 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:54:33.052474 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:54:33.052474 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:54:33.053597 (+ 1123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:54:33.053600 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:54:33.054306 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:54:33.054308 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:54:33.054413 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:33.054415 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 16:54:33.054469 (+ 54us) client_negotiation.cc:240] Negotiation successful
0808 16:54:33.054515 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":72,"mutex_wait_us":6}
I20260808 16:54:33.054610 31000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:33.051332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.3:37411 (local address 127.8.24.1:33713)
0808 16:54:33.051553 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:33.051562 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 16:54:33.051615 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:54:33.051714 (+ 99us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:54:33.051715 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:54:33.051716 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:54:33.051778 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:54:33.051779 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:54:33.052544 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:54:33.052544 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:54:33.053615 (+ 1071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:54:33.053616 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:54:33.054307 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:54:33.054307 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:54:33.054409 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:33.054414 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 16:54:33.054469 (+ 55us) client_negotiation.cc:240] Negotiation successful
0808 16:54:33.054514 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"mutex_wait_us":93,"thread_start_us":69,"threads_started":1}
I20260808 16:54:33.054867 30660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb32b7eee99149a680244b0413545ab1" candidate_uuid: "e2efec6d440c4e2391890da0bd7e3a92" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57fe315458094d32bff19b928d7f3e28" is_pre_election: true
I20260808 16:54:33.054982 30660 raft_consensus.cc:2454] T bb32b7eee99149a680244b0413545ab1 P 57fe315458094d32bff19b928d7f3e28 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2efec6d440c4e2391890da0bd7e3a92 in term 0.
I20260808 16:54:33.054968 30887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb32b7eee99149a680244b0413545ab1" candidate_uuid: "e2efec6d440c4e2391890da0bd7e3a92" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "396866376cb04432abf9245607b453c2" is_pre_election: true
I20260808 16:54:33.055076 30887 raft_consensus.cc:2454] T bb32b7eee99149a680244b0413545ab1 P 396866376cb04432abf9245607b453c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2efec6d440c4e2391890da0bd7e3a92 in term 0.
I20260808 16:54:33.055195 30332 leader_election.cc:302] T bb32b7eee99149a680244b0413545ab1 P e2efec6d440c4e2391890da0bd7e3a92 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57fe315458094d32bff19b928d7f3e28, e2efec6d440c4e2391890da0bd7e3a92; no voters:
I20260808 16:54:33.055330 30998 raft_consensus.cc:2781] T bb32b7eee99149a680244b0413545ab1 P e2efec6d440c4e2391890da0bd7e3a92 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 16:54:33.055382 30998 raft_consensus.cc:482] T bb32b7eee99149a680244b0413545ab1 P e2efec6d440c4e2391890da0bd7e3a92 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 16:54:33.055397 30998 raft_consensus.cc:3037] T bb32b7eee99149a680244b0413545ab1 P e2efec6d440c4e2391890da0bd7e3a92 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:54:33.055948 30998 raft_consensus.cc:504] T bb32b7eee99149a680244b0413545ab1 P e2efec6d440c4e2391890da0bd7e3a92 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57fe315458094d32bff19b928d7f3e28" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 37085 } } peers { permanent_uuid: "e2efec6d440c4e2391890da0bd7e3a92" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 39989 } } peers { permanent_uuid: "396866376cb04432abf9245607b453c2" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 37411 } }
I20260808 16:54:33.056054 30998 leader_election.cc:288] T bb32b7eee99149a680244b0413545ab1 P e2efec6d440c4e2391890da0bd7e3a92 [CANDIDATE]: Term 1 election: Requested vote from peers 57fe315458094d32bff19b928d7f3e28 (127.8.24.2:37085), 396866376cb04432abf9245607b453c2 (127.8.24.3:37411)
I20260808 16:54:33.056185 30887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb32b7eee99149a680244b0413545ab1" candidate_uuid: "e2efec6d440c4e2391890da0bd7e3a92" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "396866376cb04432abf9245607b453c2"
I20260808 16:54:33.056193 30660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb32b7eee99149a680244b0413545ab1" candidate_uuid: "e2efec6d440c4e2391890da0bd7e3a92" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57fe315458094d32bff19b928d7f3e28"
I20260808 16:54:33.056255 30660 raft_consensus.cc:3037] T bb32b7eee99149a680244b0413545ab1 P 57fe315458094d32bff19b928d7f3e28 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:54:33.056255 30887 raft_consensus.cc:3037] T bb32b7eee99149a680244b0413545ab1 P 396866376cb04432abf9245607b453c2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:54:33.056825 30887 raft_consensus.cc:2454] T bb32b7eee99149a680244b0413545ab1 P 396866376cb04432abf9245607b453c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2efec6d440c4e2391890da0bd7e3a92 in term 1.
I20260808 16:54:33.056857 30660 raft_consensus.cc:2454] T bb32b7eee99149a680244b0413545ab1 P 57fe315458094d32bff19b928d7f3e28 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2efec6d440c4e2391890da0bd7e3a92 in term 1.
I20260808 16:54:33.056998 30334 leader_election.cc:302] T bb32b7eee99149a680244b0413545ab1 P e2efec6d440c4e2391890da0bd7e3a92 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 396866376cb04432abf9245607b453c2, e2efec6d440c4e2391890da0bd7e3a92; no voters:
I20260808 16:54:33.057114 30998 raft_consensus.cc:2781] T bb32b7eee99149a680244b0413545ab1 P e2efec6d440c4e2391890da0bd7e3a92 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:54:33.057238 30998 raft_consensus.cc:686] T bb32b7eee99149a680244b0413545ab1 P e2efec6d440c4e2391890da0bd7e3a92 [term 1 LEADER]: Becoming Leader. State: Replica: e2efec6d440c4e2391890da0bd7e3a92, State: Running, Role: LEADER
I20260808 16:54:33.057310 30998 consensus_queue.cc:237] T bb32b7eee99149a680244b0413545ab1 P e2efec6d440c4e2391890da0bd7e3a92 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57fe315458094d32bff19b928d7f3e28" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 37085 } } peers { permanent_uuid: "e2efec6d440c4e2391890da0bd7e3a92" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 39989 } } peers { permanent_uuid: "396866376cb04432abf9245607b453c2" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 37411 } }
I20260808 16:54:33.057938 30192 catalog_manager.cc:5495] T bb32b7eee99149a680244b0413545ab1 P e2efec6d440c4e2391890da0bd7e3a92 reported cstate change: term changed from 0 to 1, leader changed from <none> to e2efec6d440c4e2391890da0bd7e3a92 (127.8.24.1). New cstate: current_term: 1 leader_uuid: "e2efec6d440c4e2391890da0bd7e3a92" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57fe315458094d32bff19b928d7f3e28" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 37085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2efec6d440c4e2391890da0bd7e3a92" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 39989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "396866376cb04432abf9245607b453c2" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 37411 } health_report { overall_health: UNKNOWN } } }
I20260808 16:54:33.064379 30980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:33.062565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56874 (local address 127.8.24.1:39989)
0808 16:54:33.062600 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0808 16:54:33.062602 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:54:33.062610 (+ 8us) server_negotiation.cc:407] Connection header received
0808 16:54:33.062682 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:54:33.062683 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:54:33.062696 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:54:33.062752 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 16:54:33.062837 (+ 85us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:33.063240 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:33.063865 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:33.064023 (+ 158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:33.064063 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:33.064152 (+ 89us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 16:54:33.064237 (+ 85us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 16:54:33.064284 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0808 16:54:33.064297 (+ 13us) server_negotiation.cc:299] Negotiation successful
0808 16:54:33.064309 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 16:54:33.065716 30660 raft_consensus.cc:1261] T bb32b7eee99149a680244b0413545ab1 P 57fe315458094d32bff19b928d7f3e28 [term 1 FOLLOWER]: Refusing update from remote peer e2efec6d440c4e2391890da0bd7e3a92: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:54:33.065768 30887 raft_consensus.cc:1261] T bb32b7eee99149a680244b0413545ab1 P 396866376cb04432abf9245607b453c2 [term 1 FOLLOWER]: Refusing update from remote peer e2efec6d440c4e2391890da0bd7e3a92: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:54:33.065974 30998 consensus_queue.cc:1035] T bb32b7eee99149a680244b0413545ab1 P e2efec6d440c4e2391890da0bd7e3a92 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57fe315458094d32bff19b928d7f3e28" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 37085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:54:33.066092 31003 consensus_queue.cc:1035] T bb32b7eee99149a680244b0413545ab1 P e2efec6d440c4e2391890da0bd7e3a92 [LEADER]: Connected to new peer: Peer: permanent_uuid: "396866376cb04432abf9245607b453c2" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 37411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:54:33.068136 31008 mvcc.cc:205] Tried to move back new op lower bound from 7316308267275059200 to 7316308267243208704. Current Snapshot: MvccSnapshot[applied={T|T < 7316308267275059200}]
I20260808 16:54:33.068215 31009 mvcc.cc:205] Tried to move back new op lower bound from 7316308267275059200 to 7316308267243208704. Current Snapshot: MvccSnapshot[applied={T|T < 7316308267275059200}]
I20260808 16:54:33.068462 31010 mvcc.cc:205] Tried to move back new op lower bound from 7316308267275059200 to 7316308267243208704. Current Snapshot: MvccSnapshot[applied={T|T < 7316308267275059200}]
I20260808 16:54:33.069911 30192 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57874:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 16:54:33.069988 30192 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57874:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 16:54:33.070583 30192 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 18e9d95d961846578aecef36a0092b32: Sending DeleteTablet for 3 replicas of tablet bb32b7eee99149a680244b0413545ab1
I20260808 16:54:33.070894 30394 tablet_service.cc:1514] Processing DeleteTablet for tablet bb32b7eee99149a680244b0413545ab1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:54:33 UTC) from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:56872
I20260808 16:54:33.070910 30853 tablet_service.cc:1514] Processing DeleteTablet for tablet bb32b7eee99149a680244b0413545ab1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:54:33 UTC) from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:33240
I20260808 16:54:33.070988 30635 tablet_service.cc:1514] Processing DeleteTablet for tablet bb32b7eee99149a680244b0413545ab1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:54:33 UTC) from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:51466
I20260808 16:54:33.071102 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 30299
I20260808 16:54:33.071206 31025 tablet_replica.cc:330] T bb32b7eee99149a680244b0413545ab1 P 57fe315458094d32bff19b928d7f3e28: stopping tablet replica
I20260808 16:54:33.071293 31025 raft_consensus.cc:2229] T bb32b7eee99149a680244b0413545ab1 P 57fe315458094d32bff19b928d7f3e28 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 16:54:33.071377 31025 raft_consensus.cc:2258] T bb32b7eee99149a680244b0413545ab1 P 57fe315458094d32bff19b928d7f3e28 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:54:33.078267 31024 tablet_replica.cc:330] T bb32b7eee99149a680244b0413545ab1 P 396866376cb04432abf9245607b453c2: stopping tablet replica
I20260808 16:54:33.078363 31024 raft_consensus.cc:2229] T bb32b7eee99149a680244b0413545ab1 P 396866376cb04432abf9245607b453c2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 16:54:33.078435 31024 raft_consensus.cc:2258] T bb32b7eee99149a680244b0413545ab1 P 396866376cb04432abf9245607b453c2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:54:33.079978 31024 ts_tablet_manager.cc:1905] T bb32b7eee99149a680244b0413545ab1 P 396866376cb04432abf9245607b453c2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:54:33.080775 31025 ts_tablet_manager.cc:1905] T bb32b7eee99149a680244b0413545ab1 P 57fe315458094d32bff19b928d7f3e28: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:54:33.081259 31024 ts_tablet_manager.cc:1918] T bb32b7eee99149a680244b0413545ab1 P 396866376cb04432abf9245607b453c2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 16:54:33.081311 31024 log.cc:1196] T bb32b7eee99149a680244b0413545ab1 P 396866376cb04432abf9245607b453c2: Deleting WAL directory at /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-2/wal/wals/bb32b7eee99149a680244b0413545ab1
I20260808 16:54:33.081516 31024 ts_tablet_manager.cc:1939] T bb32b7eee99149a680244b0413545ab1 P 396866376cb04432abf9245607b453c2: Deleting consensus metadata
I20260808 16:54:33.081813 30162 catalog_manager.cc:4841] TS 396866376cb04432abf9245607b453c2 (127.8.24.3:37411): tablet bb32b7eee99149a680244b0413545ab1 (table test-table [id=b382a25050b94de081af3e577a5f90fb]) successfully deleted
I20260808 16:54:33.082613 31025 ts_tablet_manager.cc:1918] T bb32b7eee99149a680244b0413545ab1 P 57fe315458094d32bff19b928d7f3e28: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 16:54:33.082691 31025 log.cc:1196] T bb32b7eee99149a680244b0413545ab1 P 57fe315458094d32bff19b928d7f3e28: Deleting WAL directory at /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-1/wal/wals/bb32b7eee99149a680244b0413545ab1
I20260808 16:54:33.083029 31025 ts_tablet_manager.cc:1939] T bb32b7eee99149a680244b0413545ab1 P 57fe315458094d32bff19b928d7f3e28: Deleting consensus metadata
I20260808 16:54:33.083678 30161 catalog_manager.cc:4841] TS 57fe315458094d32bff19b928d7f3e28 (127.8.24.2:37085): tablet bb32b7eee99149a680244b0413545ab1 (table test-table [id=b382a25050b94de081af3e577a5f90fb]) successfully deleted
W20260808 16:54:33.089449 30165 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.24.1:39989 (error 108)
I20260808 16:54:33.089653 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 30516
I20260808 16:54:33.091028 30978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:33.089758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:39989 (local address 127.0.0.1:56878)
0808 16:54:33.090957 (+ 1199us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:33.090999 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.1:39989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1177}
W20260808 16:54:33.093590 30165 catalog_manager.cc:4584] TS e2efec6d440c4e2391890da0bd7e3a92 (127.8.24.1:39989): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet bb32b7eee99149a680244b0413545ab1: Network error: Client connection negotiation failed: client connection to 127.8.24.1:39989: connect: Connection refused (error 111)
I20260808 16:54:33.103713 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 30749
I20260808 16:54:33.118197 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 30132
I20260808 16:54:33.128805 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:35859
--webserver_interface=127.8.24.62
--webserver_port=42833
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:46527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:35859
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:54:33.213285 31028 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:54:33.213459 31028 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:54:33.213479 31028 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:54:33.214540 31028 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 16:54:33.214572 31028 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:54:33.214584 31028 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:54:33.214596 31028 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 16:54:33.214609 31028 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 16:54:33.215966 31028 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:46527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:35859
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:35859
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.24.62
--webserver_port=42833
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31028
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:54:33.216135 31028 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:54:33.216296 31028 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:54:33.224085 31036 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:54:33.224090 31038 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:54:33.224110 31035 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:54:33.224309 31028 server_base.cc:1034] running on GCE node
I20260808 16:54:33.224534 31028 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:54:33.224783 31028 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:54:33.225914 31028 hybrid_clock.cc:648] HybridClock initialized: now 1786208073225903 us; error 29 us; skew 500 ppm
Aug 08 16:54:33 dist-test-slave-6zbq krb5kdc[30116](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208073, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:54:33.232613 31028 init.cc:377] Logged in from keytab as kudu/127.8.24.62@KRBTEST.COM (short username kudu)
I20260808 16:54:33.233016 31028 webserver.cc:460] Webserver started at http://127.8.24.62:42833/ using document root <none> and password file <none>
I20260808 16:54:33.233186 31028 fs_manager.cc:359] Metadata directory not provided
I20260808 16:54:33.233237 31028 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:54:33.234426 31028 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 16:54:33.234980 31051 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:54:33.235126 31028 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 16:54:33.235203 31028 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/master-0/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/master-0/wal
uuid: "18e9d95d961846578aecef36a0092b32"
format_stamp: "Formatted at 2026-08-08 16:54:32 on dist-test-slave-6zbq"
server_key: "b5b7b2e6fa7dfe87b7a3616d9f8fa617"
server_key_iv: "43423a4e4baf6068c563f9de2761e272"
server_key_version: "encryptionkey@0"
I20260808 16:54:33.235533 31028 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:54:33.252923 31028 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:54:33.254515 31028 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:54:33.254630 31028 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:54:33.258235 31028 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.62:35859
I20260808 16:54:33.258280 31162 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.62:35859 every 8 connection(s)
I20260808 16:54:33.258608 31028 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
I20260808 16:54:33.260059 31164 sys_catalog.cc:263] Verifying existing consensus state
I20260808 16:54:33.260648 31164 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 18e9d95d961846578aecef36a0092b32: Bootstrap starting.
I20260808 16:54:33.264135 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 31028
I20260808 16:54:33.264467 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.1:39989
--local_ip_for_outbound_sockets=127.8.24.1
--tserver_master_addrs=127.8.24.62:35859
--webserver_port=33289
--webserver_interface=127.8.24.1
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.1.keytab
--principal=kudu/127.8.24.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:46527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 16:54:33.268918 31164 log.cc:824] T 00000000000000000000000000000000 P 18e9d95d961846578aecef36a0092b32: Log is configured to *not* fsync() on all Append() calls
I20260808 16:54:33.270720 31164 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 18e9d95d961846578aecef36a0092b32: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 16:54:33.270941 31164 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 18e9d95d961846578aecef36a0092b32: Bootstrap complete.
I20260808 16:54:33.272912 31164 raft_consensus.cc:348] T 00000000000000000000000000000000 P 18e9d95d961846578aecef36a0092b32 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18e9d95d961846578aecef36a0092b32" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35859 } }
I20260808 16:54:33.273114 31164 raft_consensus.cc:729] T 00000000000000000000000000000000 P 18e9d95d961846578aecef36a0092b32 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18e9d95d961846578aecef36a0092b32, State: Initialized, Role: FOLLOWER
I20260808 16:54:33.273244 31164 consensus_queue.cc:260] T 00000000000000000000000000000000 P 18e9d95d961846578aecef36a0092b32 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18e9d95d961846578aecef36a0092b32" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35859 } }
I20260808 16:54:33.273301 31164 raft_consensus.cc:388] T 00000000000000000000000000000000 P 18e9d95d961846578aecef36a0092b32 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 16:54:33.273332 31164 raft_consensus.cc:482] T 00000000000000000000000000000000 P 18e9d95d961846578aecef36a0092b32 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 16:54:33.273377 31164 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 18e9d95d961846578aecef36a0092b32 [term 1 FOLLOWER]: Advancing to term 2
I20260808 16:54:33.274266 31164 raft_consensus.cc:504] T 00000000000000000000000000000000 P 18e9d95d961846578aecef36a0092b32 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18e9d95d961846578aecef36a0092b32" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35859 } }
I20260808 16:54:33.274371 31164 leader_election.cc:302] T 00000000000000000000000000000000 P 18e9d95d961846578aecef36a0092b32 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 18e9d95d961846578aecef36a0092b32; no voters:
I20260808 16:54:33.274518 31164 leader_election.cc:288] T 00000000000000000000000000000000 P 18e9d95d961846578aecef36a0092b32 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 16:54:33.274565 31172 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 18e9d95d961846578aecef36a0092b32 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 16:54:33.274675 31172 raft_consensus.cc:686] T 00000000000000000000000000000000 P 18e9d95d961846578aecef36a0092b32 [term 2 LEADER]: Becoming Leader. State: Replica: 18e9d95d961846578aecef36a0092b32, State: Running, Role: LEADER
I20260808 16:54:33.274765 31164 sys_catalog.cc:564] T 00000000000000000000000000000000 P 18e9d95d961846578aecef36a0092b32 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 16:54:33.274768 31172 consensus_queue.cc:237] T 00000000000000000000000000000000 P 18e9d95d961846578aecef36a0092b32 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18e9d95d961846578aecef36a0092b32" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35859 } }
I20260808 16:54:33.275055 31172 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18e9d95d961846578aecef36a0092b32 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 18e9d95d961846578aecef36a0092b32. Latest consensus state: current_term: 2 leader_uuid: "18e9d95d961846578aecef36a0092b32" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18e9d95d961846578aecef36a0092b32" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35859 } } }
I20260808 16:54:33.275115 31172 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18e9d95d961846578aecef36a0092b32 [sys.catalog]: This master's current role is: LEADER
I20260808 16:54:33.275266 31174 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18e9d95d961846578aecef36a0092b32 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "18e9d95d961846578aecef36a0092b32" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18e9d95d961846578aecef36a0092b32" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35859 } } }
I20260808 16:54:33.275349 31174 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18e9d95d961846578aecef36a0092b32 [sys.catalog]: This master's current role is: LEADER
I20260808 16:54:33.275759 31177 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 16:54:33.276448 31177 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 16:54:33.276540 31177 catalog_manager.cc:1259] Loaded cluster ID: a0523afa43d445e2afc668410927b486
I20260808 16:54:33.276571 31177 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 16:54:33.276778 31177 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260808 16:54:33.281795 31177 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260808 16:54:33.284049 31177 catalog_manager.cc:1504] Loading token signing keys...
I20260808 16:54:33.284220 31177 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 18e9d95d961846578aecef36a0092b32: Loaded TSK: 0
I20260808 16:54:33.284415 31177 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 16:54:33.346103 31167 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:54:33.346282 31167 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:54:33.346304 31167 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:54:33.347410 31167 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:54:33.347450 31167 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:54:33.347473 31167 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.1
I20260808 16:54:33.348830 31167 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:46527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.1.keytab
--principal=kudu/127.8.24.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.1:39989
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.24.1
--webserver_port=33289
--tserver_master_addrs=127.8.24.62:35859
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31167
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.1
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:54:33.349030 31167 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:54:33.349238 31167 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 16:54:33.354722 31167 server_base.cc:1034] running on GCE node
W20260808 16:54:33.354857 31211 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:54:33.354718 31215 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:54:33.354826 31208 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:54:33.355216 31167 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:54:33.355443 31167 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:54:33.356590 31167 hybrid_clock.cc:648] HybridClock initialized: now 1786208073356575 us; error 32 us; skew 500 ppm
Aug 08 16:54:33 dist-test-slave-6zbq krb5kdc[30116](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208073, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:54:33.358843 31167 init.cc:377] Logged in from keytab as kudu/127.8.24.1@KRBTEST.COM (short username kudu)
I20260808 16:54:33.359261 31167 webserver.cc:460] Webserver started at http://127.8.24.1:33289/ using document root <none> and password file <none>
I20260808 16:54:33.359467 31167 fs_manager.cc:359] Metadata directory not provided
I20260808 16:54:33.359519 31167 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:54:33.360741 31167 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 16:54:33.361366 31225 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:54:33.361515 31167 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 16:54:33.361582 31167 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-0/wal
uuid: "e2efec6d440c4e2391890da0bd7e3a92"
format_stamp: "Formatted at 2026-08-08 16:54:32 on dist-test-slave-6zbq"
server_key: "1c6cf7513357174cb09007eca6f374ec"
server_key_iv: "ae6589c1f3ec02c93e70e6a5758f59d9"
server_key_version: "encryptionkey@0"
I20260808 16:54:33.361871 31167 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:54:33.373217 31167 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:54:33.375272 31167 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:54:33.375389 31167 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:54:33.375612 31167 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:54:33.376108 31239 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 16:54:33.377602 31167 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 16:54:33.377652 31167 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260808 16:54:33.377678 31167 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 16:54:33.378304 31167 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 16:54:33.378399 31167 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.001s
I20260808 16:54:33.378441 31239 tablet_bootstrap.cc:492] T bb32b7eee99149a680244b0413545ab1 P e2efec6d440c4e2391890da0bd7e3a92: Bootstrap starting.
I20260808 16:54:33.385638 31167 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.1:39989
I20260808 16:54:33.385761 31414 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.1:39989 every 8 connection(s)
I20260808 16:54:33.386106 31167 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-0/data/info.pb
I20260808 16:54:33.388782 31239 log.cc:824] T bb32b7eee99149a680244b0413545ab1 P e2efec6d440c4e2391890da0bd7e3a92: Log is configured to *not* fsync() on all Append() calls
I20260808 16:54:33.389714 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 31167
I20260808 16:54:33.389998 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.2:37085
--local_ip_for_outbound_sockets=127.8.24.2
--tserver_master_addrs=127.8.24.62:35859
--webserver_port=45533
--webserver_interface=127.8.24.2
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.2.keytab
--principal=kudu/127.8.24.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:46527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 16:54:33.391299 31239 tablet_bootstrap.cc:492] T bb32b7eee99149a680244b0413545ab1 P e2efec6d440c4e2391890da0bd7e3a92: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 16:54:33.391551 31239 tablet_bootstrap.cc:492] T bb32b7eee99149a680244b0413545ab1 P e2efec6d440c4e2391890da0bd7e3a92: Bootstrap complete.
I20260808 16:54:33.392551 31239 ts_tablet_manager.cc:1397] T bb32b7eee99149a680244b0413545ab1 P e2efec6d440c4e2391890da0bd7e3a92: Time spent bootstrapping tablet: real 0.014s user 0.008s sys 0.004s
I20260808 16:54:33.393792 31239 raft_consensus.cc:348] T bb32b7eee99149a680244b0413545ab1 P e2efec6d440c4e2391890da0bd7e3a92 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57fe315458094d32bff19b928d7f3e28" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 37085 } } peers { permanent_uuid: "e2efec6d440c4e2391890da0bd7e3a92" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 39989 } } peers { permanent_uuid: "396866376cb04432abf9245607b453c2" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 37411 } }
I20260808 16:54:33.394095 31239 raft_consensus.cc:729] T bb32b7eee99149a680244b0413545ab1 P e2efec6d440c4e2391890da0bd7e3a92 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2efec6d440c4e2391890da0bd7e3a92, State: Initialized, Role: FOLLOWER
I20260808 16:54:33.394212 31239 consensus_queue.cc:260] T bb32b7eee99149a680244b0413545ab1 P e2efec6d440c4e2391890da0bd7e3a92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57fe315458094d32bff19b928d7f3e28" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 37085 } } peers { permanent_uuid: "e2efec6d440c4e2391890da0bd7e3a92" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 39989 } } peers { permanent_uuid: "396866376cb04432abf9245607b453c2" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 37411 } }
I20260808 16:54:33.394471 31239 ts_tablet_manager.cc:1428] T bb32b7eee99149a680244b0413545ab1 P e2efec6d440c4e2391890da0bd7e3a92: Time spent starting tablet: real 0.002s user 0.001s sys 0.003s
Aug 08 16:54:33 dist-test-slave-6zbq krb5kdc[30116](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208073, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.1@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260808 16:54:33.398952 31420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:33.386767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:35859 (local address 127.8.24.1:36121)
0808 16:54:33.387292 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:33.387335 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 16:54:33.387878 (+ 543us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:54:33.388519 (+ 641us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:54:33.388523 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:54:33.388855 (+ 332us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:54:33.389115 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:54:33.389124 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:54:33.390670 (+ 1546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:54:33.390672 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:54:33.391871 (+ 1199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:54:33.391874 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:54:33.392750 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:54:33.392751 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:54:33.392880 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:33.393377 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:54:33.393396 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:54:33.395083 (+ 1687us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:54:33.397171 (+ 2088us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:54:33.397179 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:54:33.397181 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:54:33.397480 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:54:33.397750 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:54:33.397750 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:54:33.397751 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:54:33.397887 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:54:33.398293 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:54:33.398299 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:54:33.398480 (+ 181us) client_negotiation.cc:769] Sending connection context
0808 16:54:33.398589 (+ 109us) client_negotiation.cc:240] Negotiation successful
0808 16:54:33.398749 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":88,"threads_started":1}
I20260808 16:54:33.398963 31421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:33.387117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:36121 (local address 127.8.24.62:35859)
0808 16:54:33.388074 (+ 957us) server_negotiation.cc:206] Beginning negotiation
0808 16:54:33.388083 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 16:54:33.388135 (+ 52us) server_negotiation.cc:407] Connection header received
0808 16:54:33.388265 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:54:33.388270 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:54:33.388323 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:54:33.388449 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:54:33.389223 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:33.390550 (+ 1327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:33.391969 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:33.392534 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:33.392614 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:33.395219 (+ 2605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:54:33.395223 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:54:33.395224 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:54:33.395235 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:54:33.397072 (+ 1837us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:54:33.397563 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:54:33.397566 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:54:33.397566 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:54:33.397679 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:54:33.397952 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:54:33.397952 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:54:33.397953 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:54:33.398221 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:54:33.398298 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 16:54:33.398668 (+ 370us) server_negotiation.cc:299] Negotiation successful
0808 16:54:33.398797 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":907,"thread_start_us":71,"threads_started":1}
I20260808 16:54:33.399855 31417 heartbeater.cc:347] Connected to a master server at 127.8.24.62:35859
I20260808 16:54:33.399951 31417 heartbeater.cc:464] Registering TS with master...
I20260808 16:54:33.400130 31417 heartbeater.cc:511] Master 127.8.24.62:35859 requested a full tablet report, sending...
I20260808 16:54:33.400839 31077 ts_manager.cc:194] Registered new tserver with Master: e2efec6d440c4e2391890da0bd7e3a92 (127.8.24.1:39989)
I20260808 16:54:33.401892 31077 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.1@KRBTEST.COM'} at 127.8.24.1:36121
I20260808 16:54:33.402822 31417 heartbeater.cc:503] Master 127.8.24.62:35859 was elected leader, sending a full tablet report...
Aug 08 16:54:33 dist-test-slave-6zbq krb5kdc[30116](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208073, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for kudu/127.8.24.1@KRBTEST.COM
I20260808 16:54:33.408495 31428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:33.401544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56882 (local address 127.8.24.1:39989)
0808 16:54:33.401670 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0808 16:54:33.401674 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 16:54:33.401683 (+ 9us) server_negotiation.cc:407] Connection header received
0808 16:54:33.401712 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:54:33.401714 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:54:33.401725 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:54:33.401790 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:54:33.402246 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:33.402989 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:33.403747 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:33.404046 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:33.404111 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:33.405928 (+ 1817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:54:33.405935 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:54:33.405935 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:54:33.405941 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:54:33.407426 (+ 1485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:54:33.407822 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:54:33.407824 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:54:33.407824 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:54:33.407871 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:54:33.408041 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:54:33.408042 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:54:33.408042 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:54:33.408150 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:54:33.408202 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 16:54:33.408368 (+ 166us) server_negotiation.cc:299] Negotiation successful
0808 16:54:33.408434 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":61,"threads_started":1}
I20260808 16:54:33.408536 31427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:33.401364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:39989 (local address 127.0.0.1:56882)
0808 16:54:33.401520 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:33.401559 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 16:54:33.401623 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:54:33.401805 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:54:33.401809 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:54:33.402120 (+ 311us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:54:33.402171 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:54:33.402173 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:54:33.403060 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:54:33.403061 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:54:33.403670 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:54:33.403672 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:54:33.404119 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:54:33.404123 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:54:33.404208 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:33.404591 (+ 383us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:54:33.404603 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:54:33.405854 (+ 1251us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:54:33.407498 (+ 1644us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:54:33.407506 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:54:33.407508 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:54:33.407750 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:54:33.407923 (+ 173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:54:33.407923 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:54:33.407924 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:54:33.407987 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:54:33.408199 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:54:33.408205 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:54:33.408261 (+ 56us) client_negotiation.cc:769] Sending connection context
0808 16:54:33.408336 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 16:54:33.408443 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":80,"threads_started":1}
I20260808 16:54:33.409053 31300 tablet_service.cc:1514] Processing DeleteTablet for tablet bb32b7eee99149a680244b0413545ab1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:54:33 UTC) from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:56882
I20260808 16:54:33.409519 31429 tablet_replica.cc:330] stopping tablet replica
I20260808 16:54:33.409615 31429 raft_consensus.cc:2229] T bb32b7eee99149a680244b0413545ab1 P e2efec6d440c4e2391890da0bd7e3a92 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 16:54:33.409673 31429 raft_consensus.cc:2258] T bb32b7eee99149a680244b0413545ab1 P e2efec6d440c4e2391890da0bd7e3a92 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:54:33.409924 31429 ts_tablet_manager.cc:1905] T bb32b7eee99149a680244b0413545ab1 P e2efec6d440c4e2391890da0bd7e3a92: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:54:33.412063 31429 ts_tablet_manager.cc:1918] T bb32b7eee99149a680244b0413545ab1 P e2efec6d440c4e2391890da0bd7e3a92: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 16:54:33.412134 31429 log.cc:1196] T bb32b7eee99149a680244b0413545ab1 P e2efec6d440c4e2391890da0bd7e3a92: Deleting WAL directory at /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-0/wal/wals/bb32b7eee99149a680244b0413545ab1
I20260808 16:54:33.412352 31429 ts_tablet_manager.cc:1939] T bb32b7eee99149a680244b0413545ab1 P e2efec6d440c4e2391890da0bd7e3a92: Deleting consensus metadata
I20260808 16:54:33.412797 31055 catalog_manager.cc:4841] TS e2efec6d440c4e2391890da0bd7e3a92 (127.8.24.1:39989): tablet bb32b7eee99149a680244b0413545ab1 (table test-table [id=b382a25050b94de081af3e577a5f90fb]) successfully deleted
W20260808 16:54:33.472949 31425 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:54:33.473109 31425 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:54:33.473130 31425 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:54:33.474213 31425 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:54:33.474252 31425 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:54:33.474277 31425 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.2
I20260808 16:54:33.475733 31425 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:46527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.2.keytab
--principal=kudu/127.8.24.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.2:37085
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.24.2
--webserver_port=45533
--tserver_master_addrs=127.8.24.62:35859
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31425
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.2
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:54:33.475921 31425 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:54:33.476125 31425 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:54:33.480578 31436 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:54:33.480600 31439 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:54:33.480643 31425 server_base.cc:1034] running on GCE node
W20260808 16:54:33.480611 31437 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:54:33.480893 31425 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:54:33.481070 31425 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:54:33.482208 31425 hybrid_clock.cc:648] HybridClock initialized: now 1786208073482198 us; error 25 us; skew 500 ppm
Aug 08 16:54:33 dist-test-slave-6zbq krb5kdc[30116](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208073, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:54:33.484354 31425 init.cc:377] Logged in from keytab as kudu/127.8.24.2@KRBTEST.COM (short username kudu)
I20260808 16:54:33.484756 31425 webserver.cc:460] Webserver started at http://127.8.24.2:45533/ using document root <none> and password file <none>
I20260808 16:54:33.484956 31425 fs_manager.cc:359] Metadata directory not provided
I20260808 16:54:33.485013 31425 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:54:33.486161 31425 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 16:54:33.486698 31449 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:54:33.486836 31425 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 16:54:33.486904 31425 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-1/wal
uuid: "57fe315458094d32bff19b928d7f3e28"
format_stamp: "Formatted at 2026-08-08 16:54:32 on dist-test-slave-6zbq"
server_key: "2778c3b10f02cc8146e641346f5e74ab"
server_key_iv: "1a2dd2c36087f0ba9f98d3afcd71d2bf"
server_key_version: "encryptionkey@0"
I20260808 16:54:33.487174 31425 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:54:33.508558 31425 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:54:33.510437 31425 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:54:33.510560 31425 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:54:33.510751 31425 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:54:33.511004 31425 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:54:33.511036 31425 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:54:33.511061 31425 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:54:33.511080 31425 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:54:33.516294 31425 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.2:37085
I20260808 16:54:33.516330 31640 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.2:37085 every 8 connection(s)
I20260808 16:54:33.516603 31425 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-1/data/info.pb
I20260808 16:54:33.517045 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 31425
I20260808 16:54:33.517285 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.3:37411
--local_ip_for_outbound_sockets=127.8.24.3
--tserver_master_addrs=127.8.24.62:35859
--webserver_port=44891
--webserver_interface=127.8.24.3
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.3.keytab
--principal=kudu/127.8.24.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:46527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 16:54:33 dist-test-slave-6zbq krb5kdc[30116](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208073, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.2@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260808 16:54:33.527238 31421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:33.518761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:40429 (local address 127.8.24.62:35859)
0808 16:54:33.518804 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 16:54:33.518807 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 16:54:33.519009 (+ 202us) server_negotiation.cc:407] Connection header received
0808 16:54:33.519415 (+ 406us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:54:33.519416 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:54:33.519428 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:54:33.519495 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:54:33.520281 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:33.520771 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:33.522185 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:33.522401 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:33.522445 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:33.524686 (+ 2241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:54:33.524688 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:54:33.524688 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:54:33.524696 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:54:33.525912 (+ 1216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:54:33.526379 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:54:33.526380 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:54:33.526381 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:54:33.526428 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:54:33.526674 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:54:33.526675 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:54:33.526675 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:54:33.526775 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:54:33.526819 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0808 16:54:33.527112 (+ 293us) server_negotiation.cc:299] Negotiation successful
0808 16:54:33.527175 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 16:54:33.527484 31645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:33.518752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:35859 (local address 127.8.24.2:40429)
0808 16:54:33.518898 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:33.518934 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 16:54:33.519335 (+ 401us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:54:33.519542 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:54:33.519545 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:54:33.520037 (+ 492us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:54:33.520205 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:54:33.520209 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:54:33.520864 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:54:33.520865 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:54:33.522098 (+ 1233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:54:33.522102 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:54:33.522477 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:54:33.522478 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:54:33.522613 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:33.523026 (+ 413us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:54:33.523036 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:54:33.524587 (+ 1551us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:54:33.525989 (+ 1402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:54:33.525996 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:54:33.525998 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:54:33.526288 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:54:33.526485 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:54:33.526486 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:54:33.526486 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:54:33.526615 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:54:33.526830 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:54:33.526836 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:54:33.526990 (+ 154us) client_negotiation.cc:769] Sending connection context
0808 16:54:33.527106 (+ 116us) client_negotiation.cc:240] Negotiation successful
0808 16:54:33.527271 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":67,"threads_started":1}
I20260808 16:54:33.527951 31642 heartbeater.cc:347] Connected to a master server at 127.8.24.62:35859
I20260808 16:54:33.528047 31642 heartbeater.cc:464] Registering TS with master...
I20260808 16:54:33.528240 31642 heartbeater.cc:511] Master 127.8.24.62:35859 requested a full tablet report, sending...
I20260808 16:54:33.528599 31077 ts_manager.cc:194] Registered new tserver with Master: 57fe315458094d32bff19b928d7f3e28 (127.8.24.2:37085)
I20260808 16:54:33.528947 31077 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.2@KRBTEST.COM'} at 127.8.24.2:40429
W20260808 16:54:33.599574 31644 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:54:33.599743 31644 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:54:33.599764 31644 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:54:33.600842 31644 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:54:33.600880 31644 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:54:33.600908 31644 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.3
I20260808 16:54:33.602296 31644 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:46527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.3.keytab
--principal=kudu/127.8.24.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.3:37411
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.24.3
--webserver_port=44891
--tserver_master_addrs=127.8.24.62:35859
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31644
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.3
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:54:33.602525 31644 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:54:33.602717 31644 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:54:33.607183 31657 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:54:33.607196 31656 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:54:33.607275 31644 server_base.cc:1034] running on GCE node
W20260808 16:54:33.607196 31661 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:54:33.607577 31644 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:54:33.607795 31644 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:54:33.608930 31644 hybrid_clock.cc:648] HybridClock initialized: now 1786208073608916 us; error 28 us; skew 500 ppm
Aug 08 16:54:33 dist-test-slave-6zbq krb5kdc[30116](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208073, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:54:33.611085 31644 init.cc:377] Logged in from keytab as kudu/127.8.24.3@KRBTEST.COM (short username kudu)
I20260808 16:54:33.611449 31644 webserver.cc:460] Webserver started at http://127.8.24.3:44891/ using document root <none> and password file <none>
I20260808 16:54:33.611634 31644 fs_manager.cc:359] Metadata directory not provided
I20260808 16:54:33.611694 31644 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:54:33.612867 31644 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 16:54:33.613350 31672 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:54:33.613477 31644 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 16:54:33.613555 31644 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-2/wal
uuid: "396866376cb04432abf9245607b453c2"
format_stamp: "Formatted at 2026-08-08 16:54:32 on dist-test-slave-6zbq"
server_key: "3345e3d822c4b152c4f06343904934b9"
server_key_iv: "ad80ce4eda7303fb9501cc055e140da7"
server_key_version: "encryptionkey@0"
I20260808 16:54:33.613834 31644 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:54:33.627193 31644 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:54:33.628670 31644 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:54:33.628788 31644 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:54:33.628965 31644 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:54:33.629208 31644 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:54:33.629240 31644 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:54:33.629263 31644 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:54:33.629292 31644 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:54:33.634505 31644 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.3:37411
I20260808 16:54:33.634555 31857 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.3:37411 every 8 connection(s)
I20260808 16:54:33.634835 31644 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-2/data/info.pb
Aug 08 16:54:33 dist-test-slave-6zbq krb5kdc[30116](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208073, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.3@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260808 16:54:33.642432 31421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:33.635409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:41415 (local address 127.8.24.62:35859)
0808 16:54:33.635453 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 16:54:33.635455 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:54:33.635646 (+ 191us) server_negotiation.cc:407] Connection header received
0808 16:54:33.636047 (+ 401us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:54:33.636048 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:54:33.636057 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:54:33.636103 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:54:33.636899 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:33.637390 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:33.638304 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:33.638521 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:33.638570 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:33.640293 (+ 1723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:54:33.640295 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:54:33.640295 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:54:33.640302 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:54:33.641482 (+ 1180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:54:33.641774 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:54:33.641774 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:54:33.641775 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:54:33.641818 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:54:33.641977 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:54:33.641977 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:54:33.641978 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:54:33.642063 (+ 85us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:54:33.642104 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0808 16:54:33.642289 (+ 185us) server_negotiation.cc:299] Negotiation successful
0808 16:54:33.642368 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 16:54:33.642580 31863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:33.635394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:35859 (local address 127.8.24.3:41415)
0808 16:54:33.635539 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:33.635569 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0808 16:54:33.635976 (+ 407us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:54:33.636155 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:54:33.636158 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:54:33.636658 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:54:33.636827 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:54:33.636831 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:54:33.637460 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:54:33.637460 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:54:33.638231 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:54:33.638233 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:54:33.638582 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:54:33.638583 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:54:33.638694 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:33.639047 (+ 353us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:54:33.639055 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:54:33.640215 (+ 1160us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:54:33.641539 (+ 1324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:54:33.641541 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:54:33.641543 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:54:33.641724 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:54:33.641859 (+ 135us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:54:33.641859 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:54:33.641860 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:54:33.641937 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:54:33.642102 (+ 165us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:54:33.642105 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:54:33.642199 (+ 94us) client_negotiation.cc:769] Sending connection context
0808 16:54:33.642270 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 16:54:33.642403 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":78,"threads_started":1}
I20260808 16:54:33.642910 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 31644
I20260808 16:54:33.643056 31858 heartbeater.cc:347] Connected to a master server at 127.8.24.62:35859
I20260808 16:54:33.643148 31858 heartbeater.cc:464] Registering TS with master...
I20260808 16:54:33.643326 31858 heartbeater.cc:511] Master 127.8.24.62:35859 requested a full tablet report, sending...
I20260808 16:54:33.643662 31077 ts_manager.cc:194] Registered new tserver with Master: 396866376cb04432abf9245607b453c2 (127.8.24.3:37411)
I20260808 16:54:33.644003 31077 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.3@KRBTEST.COM'} at 127.8.24.3:41415
I20260808 16:54:33.648264 31421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:33.643224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57884 (local address 127.8.24.62:35859)
0808 16:54:33.643261 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 16:54:33.643263 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:54:33.643410 (+ 147us) server_negotiation.cc:407] Connection header received
0808 16:54:33.643431 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:54:33.643432 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:54:33.643440 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:54:33.643485 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:54:33.644053 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:33.644514 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:33.645193 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:33.645381 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:33.645431 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:33.646411 (+ 980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:54:33.646412 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:54:33.646412 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:54:33.646419 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:54:33.647344 (+ 925us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:54:33.647634 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:54:33.647635 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:54:33.647635 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:54:33.647685 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:54:33.647841 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:54:33.647841 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:54:33.647842 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:54:33.647937 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:54:33.648078 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0808 16:54:33.648123 (+ 45us) server_negotiation.cc:299] Negotiation successful
0808 16:54:33.648190 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 16:54:33.648816 8288 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 16:54:33.651237 31421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:33.649084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57890 (local address 127.8.24.62:35859)
0808 16:54:33.649139 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 16:54:33.649140 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 16:54:33.649211 (+ 71us) server_negotiation.cc:407] Connection header received
0808 16:54:33.649277 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:54:33.649278 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:54:33.649311 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:54:33.649368 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 16:54:33.649462 (+ 94us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:33.649881 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:33.650612 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:33.650801 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:33.650851 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:33.650995 (+ 144us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 16:54:33.651078 (+ 83us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 16:54:33.651150 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 16:54:33.651169 (+ 19us) server_negotiation.cc:299] Negotiation successful
0808 16:54:33.651184 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 16:54:33.651491 31077 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:57890:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 16:54:33.652915 31077 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 16:54:33.654775 31300 tablet_service.cc:1467] Processing CreateTablet for tablet b5c8291ee75346bfbd3a66ae1077d5fb (DEFAULT_TABLE table=test-table [id=b02fec44defb4bbaacd432017cc64a36]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:54:33.654963 31300 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b5c8291ee75346bfbd3a66ae1077d5fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:54:33.656523 31239 tablet_bootstrap.cc:492] T b5c8291ee75346bfbd3a66ae1077d5fb P e2efec6d440c4e2391890da0bd7e3a92: Bootstrap starting.
I20260808 16:54:33.656949 31239 tablet_bootstrap.cc:654] T b5c8291ee75346bfbd3a66ae1077d5fb P e2efec6d440c4e2391890da0bd7e3a92: Neither blocks nor log segments found. Creating new log.
I20260808 16:54:33.657478 31239 tablet_bootstrap.cc:492] T b5c8291ee75346bfbd3a66ae1077d5fb P e2efec6d440c4e2391890da0bd7e3a92: No bootstrap required, opened a new log
I20260808 16:54:33.657533 31239 ts_tablet_manager.cc:1397] T b5c8291ee75346bfbd3a66ae1077d5fb P e2efec6d440c4e2391890da0bd7e3a92: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260808 16:54:33.657850 31239 raft_consensus.cc:348] T b5c8291ee75346bfbd3a66ae1077d5fb P e2efec6d440c4e2391890da0bd7e3a92 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "396866376cb04432abf9245607b453c2" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 37411 } } peers { permanent_uuid: "57fe315458094d32bff19b928d7f3e28" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 37085 } } peers { permanent_uuid: "e2efec6d440c4e2391890da0bd7e3a92" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 39989 } }
I20260808 16:54:33.657908 31427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:33.654491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.3:37411 (local address 127.0.0.1:33252)
0808 16:54:33.654519 (+ 28us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:33.654528 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 16:54:33.654564 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:54:33.654972 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:54:33.654973 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:54:33.654975 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:54:33.655012 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:54:33.655014 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:54:33.655896 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:54:33.655897 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:54:33.656857 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:54:33.656860 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:54:33.657668 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:54:33.657669 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:54:33.657738 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:33.657756 (+ 18us) client_negotiation.cc:769] Sending connection context
0808 16:54:33.657803 (+ 47us) client_negotiation.cc:240] Negotiation successful
0808 16:54:33.657845 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15}
I20260808 16:54:33.657919 31867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:33.654658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33252 (local address 127.8.24.3:37411)
0808 16:54:33.654797 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0808 16:54:33.654802 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 16:54:33.654811 (+ 9us) server_negotiation.cc:407] Connection header received
0808 16:54:33.654842 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:54:33.654844 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:54:33.654857 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:54:33.654959 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:54:33.655084 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:33.655811 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:33.656948 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:33.657595 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:33.657653 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:33.657706 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 16:54:33.657821 (+ 115us) server_negotiation.cc:299] Negotiation successful
0808 16:54:33.657853 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":64,"threads_started":1}
I20260808 16:54:33.657922 31239 raft_consensus.cc:374] T b5c8291ee75346bfbd3a66ae1077d5fb P e2efec6d440c4e2391890da0bd7e3a92 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:54:33.657967 31239 raft_consensus.cc:729] T b5c8291ee75346bfbd3a66ae1077d5fb P e2efec6d440c4e2391890da0bd7e3a92 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2efec6d440c4e2391890da0bd7e3a92, State: Initialized, Role: FOLLOWER
I20260808 16:54:33.658015 31239 consensus_queue.cc:260] T b5c8291ee75346bfbd3a66ae1077d5fb P e2efec6d440c4e2391890da0bd7e3a92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "396866376cb04432abf9245607b453c2" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 37411 } } peers { permanent_uuid: "57fe315458094d32bff19b928d7f3e28" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 37085 } } peers { permanent_uuid: "e2efec6d440c4e2391890da0bd7e3a92" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 39989 } }
I20260808 16:54:33.658104 31239 ts_tablet_manager.cc:1428] T b5c8291ee75346bfbd3a66ae1077d5fb P e2efec6d440c4e2391890da0bd7e3a92: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 16:54:33.658527 31866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:33.654546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.2:37085 (local address 127.0.0.1:51470)
0808 16:54:33.654644 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:33.654653 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 16:54:33.654694 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:54:33.655134 (+ 440us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:54:33.655135 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:54:33.655137 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:54:33.655171 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:54:33.655172 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:54:33.656299 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:54:33.656300 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:54:33.657316 (+ 1016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:54:33.657319 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:54:33.658294 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:54:33.658295 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:54:33.658381 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:33.658385 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 16:54:33.658414 (+ 29us) client_negotiation.cc:240] Negotiation successful
0808 16:54:33.658453 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":85,"thread_start_us":72,"threads_started":1}
I20260808 16:54:33.658555 31745 tablet_service.cc:1467] Processing CreateTablet for tablet b5c8291ee75346bfbd3a66ae1077d5fb (DEFAULT_TABLE table=test-table [id=b02fec44defb4bbaacd432017cc64a36]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:54:33.658829 31868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:33.654743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51470 (local address 127.8.24.2:37085)
0808 16:54:33.654946 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0808 16:54:33.654950 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 16:54:33.654962 (+ 12us) server_negotiation.cc:407] Connection header received
0808 16:54:33.655001 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:54:33.655003 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:54:33.655053 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:54:33.655132 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:54:33.655236 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:33.656196 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:33.657385 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:33.658185 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:33.658627 (+ 442us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:33.658692 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 16:54:33.658716 (+ 24us) server_negotiation.cc:299] Negotiation successful
0808 16:54:33.658754 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":78,"threads_started":1}
I20260808 16:54:33.658865 31745 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b5c8291ee75346bfbd3a66ae1077d5fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:54:33.659389 31525 tablet_service.cc:1467] Processing CreateTablet for tablet b5c8291ee75346bfbd3a66ae1077d5fb (DEFAULT_TABLE table=test-table [id=b02fec44defb4bbaacd432017cc64a36]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:54:33.659660 31525 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b5c8291ee75346bfbd3a66ae1077d5fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:54:33.661410 31870 tablet_bootstrap.cc:492] T b5c8291ee75346bfbd3a66ae1077d5fb P 396866376cb04432abf9245607b453c2: Bootstrap starting.
I20260808 16:54:33.661942 31870 tablet_bootstrap.cc:654] T b5c8291ee75346bfbd3a66ae1077d5fb P 396866376cb04432abf9245607b453c2: Neither blocks nor log segments found. Creating new log.
I20260808 16:54:33.662144 31871 tablet_bootstrap.cc:492] T b5c8291ee75346bfbd3a66ae1077d5fb P 57fe315458094d32bff19b928d7f3e28: Bootstrap starting.
I20260808 16:54:33.662437 31870 log.cc:824] T b5c8291ee75346bfbd3a66ae1077d5fb P 396866376cb04432abf9245607b453c2: Log is configured to *not* fsync() on all Append() calls
I20260808 16:54:33.662717 31871 tablet_bootstrap.cc:654] T b5c8291ee75346bfbd3a66ae1077d5fb P 57fe315458094d32bff19b928d7f3e28: Neither blocks nor log segments found. Creating new log.
I20260808 16:54:33.663065 31870 tablet_bootstrap.cc:492] T b5c8291ee75346bfbd3a66ae1077d5fb P 396866376cb04432abf9245607b453c2: No bootstrap required, opened a new log
I20260808 16:54:33.663120 31870 ts_tablet_manager.cc:1397] T b5c8291ee75346bfbd3a66ae1077d5fb P 396866376cb04432abf9245607b453c2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 16:54:33.663111 31871 log.cc:824] T b5c8291ee75346bfbd3a66ae1077d5fb P 57fe315458094d32bff19b928d7f3e28: Log is configured to *not* fsync() on all Append() calls
I20260808 16:54:33.663745 31871 tablet_bootstrap.cc:492] T b5c8291ee75346bfbd3a66ae1077d5fb P 57fe315458094d32bff19b928d7f3e28: No bootstrap required, opened a new log
I20260808 16:54:33.663801 31871 ts_tablet_manager.cc:1397] T b5c8291ee75346bfbd3a66ae1077d5fb P 57fe315458094d32bff19b928d7f3e28: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 16:54:33.664480 31870 raft_consensus.cc:348] T b5c8291ee75346bfbd3a66ae1077d5fb P 396866376cb04432abf9245607b453c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "396866376cb04432abf9245607b453c2" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 37411 } } peers { permanent_uuid: "57fe315458094d32bff19b928d7f3e28" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 37085 } } peers { permanent_uuid: "e2efec6d440c4e2391890da0bd7e3a92" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 39989 } }
I20260808 16:54:33.664657 31870 raft_consensus.cc:374] T b5c8291ee75346bfbd3a66ae1077d5fb P 396866376cb04432abf9245607b453c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:54:33.664682 31870 raft_consensus.cc:729] T b5c8291ee75346bfbd3a66ae1077d5fb P 396866376cb04432abf9245607b453c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 396866376cb04432abf9245607b453c2, State: Initialized, Role: FOLLOWER
I20260808 16:54:33.664779 31870 consensus_queue.cc:260] T b5c8291ee75346bfbd3a66ae1077d5fb P 396866376cb04432abf9245607b453c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "396866376cb04432abf9245607b453c2" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 37411 } } peers { permanent_uuid: "57fe315458094d32bff19b928d7f3e28" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 37085 } } peers { permanent_uuid: "e2efec6d440c4e2391890da0bd7e3a92" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 39989 } }
I20260808 16:54:33.665005 31870 ts_tablet_manager.cc:1428] T b5c8291ee75346bfbd3a66ae1077d5fb P 396866376cb04432abf9245607b453c2: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 16:54:33.665043 31858 heartbeater.cc:503] Master 127.8.24.62:35859 was elected leader, sending a full tablet report...
I20260808 16:54:33.665388 31871 raft_consensus.cc:348] T b5c8291ee75346bfbd3a66ae1077d5fb P 57fe315458094d32bff19b928d7f3e28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "396866376cb04432abf9245607b453c2" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 37411 } } peers { permanent_uuid: "57fe315458094d32bff19b928d7f3e28" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 37085 } } peers { permanent_uuid: "e2efec6d440c4e2391890da0bd7e3a92" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 39989 } }
I20260808 16:54:33.665514 31871 raft_consensus.cc:374] T b5c8291ee75346bfbd3a66ae1077d5fb P 57fe315458094d32bff19b928d7f3e28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:54:33.665546 31871 raft_consensus.cc:729] T b5c8291ee75346bfbd3a66ae1077d5fb P 57fe315458094d32bff19b928d7f3e28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57fe315458094d32bff19b928d7f3e28, State: Initialized, Role: FOLLOWER
I20260808 16:54:33.665652 31871 consensus_queue.cc:260] T b5c8291ee75346bfbd3a66ae1077d5fb P 57fe315458094d32bff19b928d7f3e28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "396866376cb04432abf9245607b453c2" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 37411 } } peers { permanent_uuid: "57fe315458094d32bff19b928d7f3e28" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 37085 } } peers { permanent_uuid: "e2efec6d440c4e2391890da0bd7e3a92" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 39989 } }
I20260808 16:54:33.665915 31871 ts_tablet_manager.cc:1428] T b5c8291ee75346bfbd3a66ae1077d5fb P 57fe315458094d32bff19b928d7f3e28: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 16:54:33.665946 31642 heartbeater.cc:503] Master 127.8.24.62:35859 was elected leader, sending a full tablet report...
W20260808 16:54:33.767077 31643 tablet.cc:2369] T b5c8291ee75346bfbd3a66ae1077d5fb P 57fe315458094d32bff19b928d7f3e28: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 16:54:33.885327 31860 tablet.cc:2369] T b5c8291ee75346bfbd3a66ae1077d5fb P 396866376cb04432abf9245607b453c2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 16:54:33.887039 31419 tablet.cc:2369] T b5c8291ee75346bfbd3a66ae1077d5fb P e2efec6d440c4e2391890da0bd7e3a92: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 16:54:33.888360 31878 raft_consensus.cc:482] T b5c8291ee75346bfbd3a66ae1077d5fb P 57fe315458094d32bff19b928d7f3e28 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 16:54:33.888455 31878 raft_consensus.cc:504] T b5c8291ee75346bfbd3a66ae1077d5fb P 57fe315458094d32bff19b928d7f3e28 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "396866376cb04432abf9245607b453c2" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 37411 } } peers { permanent_uuid: "57fe315458094d32bff19b928d7f3e28" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 37085 } } peers { permanent_uuid: "e2efec6d440c4e2391890da0bd7e3a92" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 39989 } }
I20260808 16:54:33.888787 31878 leader_election.cc:288] T b5c8291ee75346bfbd3a66ae1077d5fb P 57fe315458094d32bff19b928d7f3e28 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 396866376cb04432abf9245607b453c2 (127.8.24.3:37411), e2efec6d440c4e2391890da0bd7e3a92 (127.8.24.1:39989)
I20260808 16:54:33.891458 31645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:33.888950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.3:37411 (local address 127.8.24.2:52249)
0808 16:54:33.889007 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:33.889019 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 16:54:33.889068 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:54:33.889187 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:54:33.889188 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:54:33.889189 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:54:33.889251 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:54:33.889253 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:54:33.889841 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:54:33.889842 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:54:33.890708 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:54:33.890711 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:54:33.891219 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:54:33.891220 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:54:33.891301 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:33.891304 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 16:54:33.891346 (+ 42us) client_negotiation.cc:240] Negotiation successful
0808 16:54:33.891385 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260808 16:54:33.891456 31867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:33.889033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:52249 (local address 127.8.24.3:37411)
0808 16:54:33.889073 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 16:54:33.889075 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:54:33.889083 (+ 8us) server_negotiation.cc:407] Connection header received
0808 16:54:33.889113 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:54:33.889114 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:54:33.889124 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:54:33.889173 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:54:33.889306 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:33.889770 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:33.890787 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:33.891146 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:33.891208 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:33.891219 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 16:54:33.891357 (+ 138us) server_negotiation.cc:299] Negotiation successful
0808 16:54:33.891395 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 16:54:33.891659 31428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:33.888999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:33697 (local address 127.8.24.1:39989)
0808 16:54:33.889055 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 16:54:33.889057 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:54:33.889228 (+ 171us) server_negotiation.cc:407] Connection header received
0808 16:54:33.889308 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:54:33.889309 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:54:33.889332 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:54:33.889385 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:54:33.889521 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:33.889996 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:33.890965 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:33.891354 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:33.891425 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:33.891498 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 16:54:33.891565 (+ 67us) server_negotiation.cc:299] Negotiation successful
0808 16:54:33.891599 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 16:54:33.891672 31879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:33.888950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:39989 (local address 127.8.24.2:33697)
0808 16:54:33.889172 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:33.889181 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 16:54:33.889248 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:54:33.889393 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:54:33.889394 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:54:33.889396 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:54:33.889466 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:54:33.889467 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:54:33.890079 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:54:33.890080 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:54:33.890883 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:54:33.890886 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:54:33.891439 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:54:33.891439 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:54:33.891509 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:33.891513 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 16:54:33.891561 (+ 48us) client_negotiation.cc:240] Negotiation successful
0808 16:54:33.891596 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205,"mutex_wait_us":74,"thread_start_us":86,"threads_started":1}
I20260808 16:54:33.891788 31780 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5c8291ee75346bfbd3a66ae1077d5fb" candidate_uuid: "57fe315458094d32bff19b928d7f3e28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "396866376cb04432abf9245607b453c2" is_pre_election: true
I20260808 16:54:33.891921 31780 raft_consensus.cc:2454] T b5c8291ee75346bfbd3a66ae1077d5fb P 396866376cb04432abf9245607b453c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57fe315458094d32bff19b928d7f3e28 in term 0.
I20260808 16:54:33.891960 31338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5c8291ee75346bfbd3a66ae1077d5fb" candidate_uuid: "57fe315458094d32bff19b928d7f3e28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2efec6d440c4e2391890da0bd7e3a92" is_pre_election: true
I20260808 16:54:33.892124 31338 raft_consensus.cc:2454] T b5c8291ee75346bfbd3a66ae1077d5fb P e2efec6d440c4e2391890da0bd7e3a92 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57fe315458094d32bff19b928d7f3e28 in term 0.
I20260808 16:54:33.892130 31454 leader_election.cc:302] T b5c8291ee75346bfbd3a66ae1077d5fb P 57fe315458094d32bff19b928d7f3e28 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 396866376cb04432abf9245607b453c2, 57fe315458094d32bff19b928d7f3e28; no voters:
I20260808 16:54:33.892263 31878 raft_consensus.cc:2781] T b5c8291ee75346bfbd3a66ae1077d5fb P 57fe315458094d32bff19b928d7f3e28 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 16:54:33.892316 31878 raft_consensus.cc:482] T b5c8291ee75346bfbd3a66ae1077d5fb P 57fe315458094d32bff19b928d7f3e28 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 16:54:33.892333 31878 raft_consensus.cc:3037] T b5c8291ee75346bfbd3a66ae1077d5fb P 57fe315458094d32bff19b928d7f3e28 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:54:33.892997 31878 raft_consensus.cc:504] T b5c8291ee75346bfbd3a66ae1077d5fb P 57fe315458094d32bff19b928d7f3e28 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "396866376cb04432abf9245607b453c2" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 37411 } } peers { permanent_uuid: "57fe315458094d32bff19b928d7f3e28" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 37085 } } peers { permanent_uuid: "e2efec6d440c4e2391890da0bd7e3a92" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 39989 } }
I20260808 16:54:33.893110 31878 leader_election.cc:288] T b5c8291ee75346bfbd3a66ae1077d5fb P 57fe315458094d32bff19b928d7f3e28 [CANDIDATE]: Term 1 election: Requested vote from peers 396866376cb04432abf9245607b453c2 (127.8.24.3:37411), e2efec6d440c4e2391890da0bd7e3a92 (127.8.24.1:39989)
I20260808 16:54:33.893236 31780 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5c8291ee75346bfbd3a66ae1077d5fb" candidate_uuid: "57fe315458094d32bff19b928d7f3e28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "396866376cb04432abf9245607b453c2"
I20260808 16:54:33.893270 31338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5c8291ee75346bfbd3a66ae1077d5fb" candidate_uuid: "57fe315458094d32bff19b928d7f3e28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2efec6d440c4e2391890da0bd7e3a92"
I20260808 16:54:33.893312 31780 raft_consensus.cc:3037] T b5c8291ee75346bfbd3a66ae1077d5fb P 396866376cb04432abf9245607b453c2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:54:33.893344 31338 raft_consensus.cc:3037] T b5c8291ee75346bfbd3a66ae1077d5fb P e2efec6d440c4e2391890da0bd7e3a92 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:54:33.893956 31338 raft_consensus.cc:2454] T b5c8291ee75346bfbd3a66ae1077d5fb P e2efec6d440c4e2391890da0bd7e3a92 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57fe315458094d32bff19b928d7f3e28 in term 1.
I20260808 16:54:33.894006 31780 raft_consensus.cc:2454] T b5c8291ee75346bfbd3a66ae1077d5fb P 396866376cb04432abf9245607b453c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57fe315458094d32bff19b928d7f3e28 in term 1.
I20260808 16:54:33.894122 31456 leader_election.cc:302] T b5c8291ee75346bfbd3a66ae1077d5fb P 57fe315458094d32bff19b928d7f3e28 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57fe315458094d32bff19b928d7f3e28, e2efec6d440c4e2391890da0bd7e3a92; no voters:
I20260808 16:54:33.894220 31878 raft_consensus.cc:2781] T b5c8291ee75346bfbd3a66ae1077d5fb P 57fe315458094d32bff19b928d7f3e28 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:54:33.894332 31878 raft_consensus.cc:686] T b5c8291ee75346bfbd3a66ae1077d5fb P 57fe315458094d32bff19b928d7f3e28 [term 1 LEADER]: Becoming Leader. State: Replica: 57fe315458094d32bff19b928d7f3e28, State: Running, Role: LEADER
I20260808 16:54:33.894443 31878 consensus_queue.cc:237] T b5c8291ee75346bfbd3a66ae1077d5fb P 57fe315458094d32bff19b928d7f3e28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "396866376cb04432abf9245607b453c2" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 37411 } } peers { permanent_uuid: "57fe315458094d32bff19b928d7f3e28" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 37085 } } peers { permanent_uuid: "e2efec6d440c4e2391890da0bd7e3a92" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 39989 } }
I20260808 16:54:33.895174 31077 catalog_manager.cc:5495] T b5c8291ee75346bfbd3a66ae1077d5fb P 57fe315458094d32bff19b928d7f3e28 reported cstate change: term changed from 0 to 1, leader changed from <none> to 57fe315458094d32bff19b928d7f3e28 (127.8.24.2). New cstate: current_term: 1 leader_uuid: "57fe315458094d32bff19b928d7f3e28" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "396866376cb04432abf9245607b453c2" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 37411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57fe315458094d32bff19b928d7f3e28" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 37085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e2efec6d440c4e2391890da0bd7e3a92" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 39989 } health_report { overall_health: UNKNOWN } } }
I20260808 16:54:33.938733 31868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:33.936779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51472 (local address 127.8.24.2:37085)
0808 16:54:33.936828 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 16:54:33.936830 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:54:33.936852 (+ 22us) server_negotiation.cc:407] Connection header received
0808 16:54:33.936879 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:54:33.936880 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:54:33.936896 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:54:33.936943 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 16:54:33.937025 (+ 82us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:33.937487 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:33.938111 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:33.938290 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:33.938329 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:33.938481 (+ 152us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 16:54:33.938569 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 16:54:33.938626 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 16:54:33.938641 (+ 15us) server_negotiation.cc:299] Negotiation successful
0808 16:54:33.938656 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 16:54:33.940225 31338 raft_consensus.cc:1261] T b5c8291ee75346bfbd3a66ae1077d5fb P e2efec6d440c4e2391890da0bd7e3a92 [term 1 FOLLOWER]: Refusing update from remote peer 57fe315458094d32bff19b928d7f3e28: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:54:33.940224 31780 raft_consensus.cc:1261] T b5c8291ee75346bfbd3a66ae1077d5fb P 396866376cb04432abf9245607b453c2 [term 1 FOLLOWER]: Refusing update from remote peer 57fe315458094d32bff19b928d7f3e28: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:54:33.940500 31882 consensus_queue.cc:1035] T b5c8291ee75346bfbd3a66ae1077d5fb P 57fe315458094d32bff19b928d7f3e28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2efec6d440c4e2391890da0bd7e3a92" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 39989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:54:33.940609 31878 consensus_queue.cc:1035] T b5c8291ee75346bfbd3a66ae1077d5fb P 57fe315458094d32bff19b928d7f3e28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "396866376cb04432abf9245607b453c2" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 37411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:54:33.943574 31891 mvcc.cc:205] Tried to move back new op lower bound from 7316308270856908800 to 7316308270672121856. Current Snapshot: MvccSnapshot[applied={T|T < 7316308270856908800}]
I20260808 16:54:33.944646 31077 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:57890:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 16:54:33.944721 31077 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:57890:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 16:54:33.945351 31077 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 18e9d95d961846578aecef36a0092b32: Sending DeleteTablet for 3 replicas of tablet b5c8291ee75346bfbd3a66ae1077d5fb
I20260808 16:54:33.945660 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 31167
I20260808 16:54:33.945686 31745 tablet_service.cc:1514] Processing DeleteTablet for tablet b5c8291ee75346bfbd3a66ae1077d5fb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:54:33 UTC) from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:33252
I20260808 16:54:33.946151 31525 tablet_service.cc:1514] Processing DeleteTablet for tablet b5c8291ee75346bfbd3a66ae1077d5fb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:54:33 UTC) from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:51470
I20260808 16:54:33.949571 31909 tablet_replica.cc:330] T b5c8291ee75346bfbd3a66ae1077d5fb P 57fe315458094d32bff19b928d7f3e28: stopping tablet replica
I20260808 16:54:33.949690 31909 raft_consensus.cc:2229] T b5c8291ee75346bfbd3a66ae1077d5fb P 57fe315458094d32bff19b928d7f3e28 [term 1 LEADER]: Raft consensus shutting down.
I20260808 16:54:33.949788 31909 raft_consensus.cc:2258] T b5c8291ee75346bfbd3a66ae1077d5fb P 57fe315458094d32bff19b928d7f3e28 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:54:33.953822 31909 ts_tablet_manager.cc:1905] T b5c8291ee75346bfbd3a66ae1077d5fb P 57fe315458094d32bff19b928d7f3e28: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:54:33.955415 31909 ts_tablet_manager.cc:1918] T b5c8291ee75346bfbd3a66ae1077d5fb P 57fe315458094d32bff19b928d7f3e28: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 16:54:33.955487 31909 log.cc:1196] T b5c8291ee75346bfbd3a66ae1077d5fb P 57fe315458094d32bff19b928d7f3e28: Deleting WAL directory at /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-1/wal/wals/b5c8291ee75346bfbd3a66ae1077d5fb
I20260808 16:54:33.955754 31909 ts_tablet_manager.cc:1939] T b5c8291ee75346bfbd3a66ae1077d5fb P 57fe315458094d32bff19b928d7f3e28: Deleting consensus metadata
I20260808 16:54:33.956732 31052 catalog_manager.cc:4841] TS 57fe315458094d32bff19b928d7f3e28 (127.8.24.2:37085): tablet b5c8291ee75346bfbd3a66ae1077d5fb (table test-table [id=b02fec44defb4bbaacd432017cc64a36]) successfully deleted
I20260808 16:54:33.958249 31906 tablet_replica.cc:330] T b5c8291ee75346bfbd3a66ae1077d5fb P 396866376cb04432abf9245607b453c2: stopping tablet replica
I20260808 16:54:33.958413 31906 raft_consensus.cc:2229] T b5c8291ee75346bfbd3a66ae1077d5fb P 396866376cb04432abf9245607b453c2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 16:54:33.958562 31906 raft_consensus.cc:2258] T b5c8291ee75346bfbd3a66ae1077d5fb P 396866376cb04432abf9245607b453c2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:54:33.959322 31906 ts_tablet_manager.cc:1905] T b5c8291ee75346bfbd3a66ae1077d5fb P 396866376cb04432abf9245607b453c2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:54:33.961164 31906 ts_tablet_manager.cc:1918] T b5c8291ee75346bfbd3a66ae1077d5fb P 396866376cb04432abf9245607b453c2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 16:54:33.961268 31906 log.cc:1196] T b5c8291ee75346bfbd3a66ae1077d5fb P 396866376cb04432abf9245607b453c2: Deleting WAL directory at /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-2/wal/wals/b5c8291ee75346bfbd3a66ae1077d5fb
I20260808 16:54:33.961622 31906 ts_tablet_manager.cc:1939] T b5c8291ee75346bfbd3a66ae1077d5fb P 396866376cb04432abf9245607b453c2: Deleting consensus metadata
I20260808 16:54:33.962105 31053 catalog_manager.cc:4841] TS 396866376cb04432abf9245607b453c2 (127.8.24.3:37411): tablet b5c8291ee75346bfbd3a66ae1077d5fb (table test-table [id=b02fec44defb4bbaacd432017cc64a36]) successfully deleted
W20260808 16:54:33.967721 31055 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.24.1:39989 (error 108)
I20260808 16:54:33.968207 31866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:33.968072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:39989 (local address 127.0.0.1:56888)
0808 16:54:33.968125 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:33.968173 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.1:39989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260808 16:54:33.968219 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 31425
W20260808 16:54:33.972908 31055 catalog_manager.cc:4584] TS e2efec6d440c4e2391890da0bd7e3a92 (127.8.24.1:39989): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b5c8291ee75346bfbd3a66ae1077d5fb: Network error: Client connection negotiation failed: client connection to 127.8.24.1:39989: connect: Connection refused (error 111)
I20260808 16:54:33.982841 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 31644
I20260808 16:54:33.992331 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 31028
I20260808 16:54:34.001791 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:35859
--webserver_interface=127.8.24.62
--webserver_port=42833
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:46527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:35859
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:54:34.079936 31913 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:54:34.080112 31913 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:54:34.080132 31913 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:54:34.081277 31913 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 16:54:34.081317 31913 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:54:34.081331 31913 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:54:34.081341 31913 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 16:54:34.081354 31913 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 16:54:34.082751 31913 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:46527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:35859
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:35859
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.24.62
--webserver_port=42833
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31913
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:54:34.082930 31913 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:54:34.083086 31913 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:54:34.087327 31926 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:54:34.087378 31913 server_base.cc:1034] running on GCE node
W20260808 16:54:34.087316 31920 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:54:34.087344 31919 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:54:34.087805 31913 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:54:34.088065 31913 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:54:34.089187 31913 hybrid_clock.cc:648] HybridClock initialized: now 1786208074089179 us; error 26 us; skew 500 ppm
Aug 08 16:54:34 dist-test-slave-6zbq krb5kdc[30116](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208074, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:54:34.091424 31913 init.cc:377] Logged in from keytab as kudu/127.8.24.62@KRBTEST.COM (short username kudu)
I20260808 16:54:34.091835 31913 webserver.cc:460] Webserver started at http://127.8.24.62:42833/ using document root <none> and password file <none>
I20260808 16:54:34.091995 31913 fs_manager.cc:359] Metadata directory not provided
I20260808 16:54:34.092032 31913 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:54:34.093159 31913 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 16:54:34.093708 31936 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:54:34.093852 31913 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 16:54:34.093930 31913 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/master-0/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/master-0/wal
uuid: "18e9d95d961846578aecef36a0092b32"
format_stamp: "Formatted at 2026-08-08 16:54:32 on dist-test-slave-6zbq"
server_key: "b5b7b2e6fa7dfe87b7a3616d9f8fa617"
server_key_iv: "43423a4e4baf6068c563f9de2761e272"
server_key_version: "encryptionkey@0"
I20260808 16:54:34.094203 31913 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:54:34.103639 31913 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:54:34.105120 31913 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:54:34.105242 31913 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:54:34.108989 31913 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.62:35859
I20260808 16:54:34.109023 32024 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.62:35859 every 8 connection(s)
I20260808 16:54:34.109351 31913 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
I20260808 16:54:34.110850 32025 sys_catalog.cc:263] Verifying existing consensus state
I20260808 16:54:34.111482 32025 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 18e9d95d961846578aecef36a0092b32: Bootstrap starting.
I20260808 16:54:34.116276 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 31913
I20260808 16:54:34.116523 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.1:39989
--local_ip_for_outbound_sockets=127.8.24.1
--tserver_master_addrs=127.8.24.62:35859
--webserver_port=33289
--webserver_interface=127.8.24.1
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.1.keytab
--principal=kudu/127.8.24.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:46527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 16:54:34.119620 32025 log.cc:824] T 00000000000000000000000000000000 P 18e9d95d961846578aecef36a0092b32: Log is configured to *not* fsync() on all Append() calls
I20260808 16:54:34.121649 32025 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 18e9d95d961846578aecef36a0092b32: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 16:54:34.121905 32025 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 18e9d95d961846578aecef36a0092b32: Bootstrap complete.
I20260808 16:54:34.124082 32025 raft_consensus.cc:348] T 00000000000000000000000000000000 P 18e9d95d961846578aecef36a0092b32 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18e9d95d961846578aecef36a0092b32" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35859 } }
I20260808 16:54:34.124298 32025 raft_consensus.cc:729] T 00000000000000000000000000000000 P 18e9d95d961846578aecef36a0092b32 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18e9d95d961846578aecef36a0092b32, State: Initialized, Role: FOLLOWER
I20260808 16:54:34.124434 32025 consensus_queue.cc:260] T 00000000000000000000000000000000 P 18e9d95d961846578aecef36a0092b32 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18e9d95d961846578aecef36a0092b32" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35859 } }
I20260808 16:54:34.124492 32025 raft_consensus.cc:388] T 00000000000000000000000000000000 P 18e9d95d961846578aecef36a0092b32 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 16:54:34.124531 32025 raft_consensus.cc:482] T 00000000000000000000000000000000 P 18e9d95d961846578aecef36a0092b32 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 16:54:34.124575 32025 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 18e9d95d961846578aecef36a0092b32 [term 2 FOLLOWER]: Advancing to term 3
I20260808 16:54:34.125494 32025 raft_consensus.cc:504] T 00000000000000000000000000000000 P 18e9d95d961846578aecef36a0092b32 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18e9d95d961846578aecef36a0092b32" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35859 } }
I20260808 16:54:34.125578 32025 leader_election.cc:302] T 00000000000000000000000000000000 P 18e9d95d961846578aecef36a0092b32 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 18e9d95d961846578aecef36a0092b32; no voters:
I20260808 16:54:34.125730 32025 leader_election.cc:288] T 00000000000000000000000000000000 P 18e9d95d961846578aecef36a0092b32 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260808 16:54:34.125787 32033 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 18e9d95d961846578aecef36a0092b32 [term 3 FOLLOWER]: Leader election won for term 3
I20260808 16:54:34.125921 32033 raft_consensus.cc:686] T 00000000000000000000000000000000 P 18e9d95d961846578aecef36a0092b32 [term 3 LEADER]: Becoming Leader. State: Replica: 18e9d95d961846578aecef36a0092b32, State: Running, Role: LEADER
I20260808 16:54:34.125970 32025 sys_catalog.cc:564] T 00000000000000000000000000000000 P 18e9d95d961846578aecef36a0092b32 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 16:54:34.126052 32033 consensus_queue.cc:237] T 00000000000000000000000000000000 P 18e9d95d961846578aecef36a0092b32 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18e9d95d961846578aecef36a0092b32" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35859 } }
I20260808 16:54:34.126372 32034 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18e9d95d961846578aecef36a0092b32 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "18e9d95d961846578aecef36a0092b32" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18e9d95d961846578aecef36a0092b32" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35859 } } }
I20260808 16:54:34.126408 32033 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18e9d95d961846578aecef36a0092b32 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 18e9d95d961846578aecef36a0092b32. Latest consensus state: current_term: 3 leader_uuid: "18e9d95d961846578aecef36a0092b32" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18e9d95d961846578aecef36a0092b32" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35859 } } }
I20260808 16:54:34.126461 32034 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18e9d95d961846578aecef36a0092b32 [sys.catalog]: This master's current role is: LEADER
I20260808 16:54:34.126479 32033 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18e9d95d961846578aecef36a0092b32 [sys.catalog]: This master's current role is: LEADER
I20260808 16:54:34.127733 32056 catalog_manager.cc:1259] Loaded cluster ID: a0523afa43d445e2afc668410927b486
I20260808 16:54:34.127782 32056 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 18e9d95d961846578aecef36a0092b32: loading cluster ID for follower catalog manager: success
W20260808 16:54:34.128077 32056 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 18e9d95d961846578aecef36a0092b32: acquiring CA information for follower catalog manager: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
W20260808 16:54:34.128123 32056 catalog_manager.cc:873] Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO: failed to prepare follower catalog manager, will retry
I20260808 16:54:34.128170 32045 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 16:54:34.128520 32045 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 16:54:34.128603 32045 catalog_manager.cc:1259] Loaded cluster ID: a0523afa43d445e2afc668410927b486
I20260808 16:54:34.128636 32045 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260808 16:54:34.128804 32045 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f3c89d129ad google::LogMessage::Fail() at ??:0
@ 0x7f3c89d1588a google::LogMessage::SendToLog() at ??:0
@ 0x7f3c89d124de google::LogMessage::Flush() at ??:0
@ 0x7f3c89d13a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f3c91c330f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f3c91c3c8b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f3c8bbe00d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f3c8bbd5954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f3c8ca396db start_thread at ??:0
@ 0x7f3c8881061f clone at ??:0
W20260808 16:54:34.208801 32029 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:54:34.208971 32029 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:54:34.208992 32029 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:54:34.210053 32029 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:54:34.210090 32029 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:54:34.210115 32029 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.1
I20260808 16:54:34.211515 32029 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:46527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.1.keytab
--principal=kudu/127.8.24.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.1:39989
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.24.1
--webserver_port=33289
--tserver_master_addrs=127.8.24.62:35859
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32029
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.1
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:54:34.211710 32029 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:54:34.211911 32029 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:54:34.216429 32083 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:54:34.216424 32085 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:54:34.216764 32029 server_base.cc:1034] running on GCE node
W20260808 16:54:34.216799 32087 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:54:34.216990 32029 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:54:34.217200 32029 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:54:34.218331 32029 hybrid_clock.cc:648] HybridClock initialized: now 1786208074218313 us; error 39 us; skew 500 ppm
Aug 08 16:54:34 dist-test-slave-6zbq krb5kdc[30116](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208074, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:54:34.220829 32029 init.cc:377] Logged in from keytab as kudu/127.8.24.1@KRBTEST.COM (short username kudu)
I20260808 16:54:34.221338 32029 webserver.cc:460] Webserver started at http://127.8.24.1:33289/ using document root <none> and password file <none>
I20260808 16:54:34.221612 32029 fs_manager.cc:359] Metadata directory not provided
I20260808 16:54:34.221673 32029 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:54:34.223291 32029 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 16:54:34.224032 32095 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:54:34.224196 32029 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 16:54:34.224283 32029 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-0/wal
uuid: "e2efec6d440c4e2391890da0bd7e3a92"
format_stamp: "Formatted at 2026-08-08 16:54:32 on dist-test-slave-6zbq"
server_key: "1c6cf7513357174cb09007eca6f374ec"
server_key_iv: "ae6589c1f3ec02c93e70e6a5758f59d9"
server_key_version: "encryptionkey@0"
I20260808 16:54:34.224680 32029 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:54:34.248874 32029 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:54:34.251073 32029 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:54:34.251243 32029 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:54:34.251508 32029 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:54:34.251971 32111 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 16:54:34.253294 32029 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 16:54:34.253343 32029 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.001s user 0.000s sys 0.000s
I20260808 16:54:34.253367 32029 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 16:54:34.253962 32029 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 16:54:34.254006 32029 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260808 16:54:34.254070 32111 tablet_bootstrap.cc:492] T b5c8291ee75346bfbd3a66ae1077d5fb P e2efec6d440c4e2391890da0bd7e3a92: Bootstrap starting.
I20260808 16:54:34.261215 32029 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.1:39989
I20260808 16:54:34.261404 32269 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.1:39989 every 8 connection(s)
I20260808 16:54:34.261636 32029 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-0/data/info.pb
I20260808 16:54:34.262020 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 32029
I20260808 16:54:34.262327 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.2:37085
--local_ip_for_outbound_sockets=127.8.24.2
--tserver_master_addrs=127.8.24.62:35859
--webserver_port=45533
--webserver_interface=127.8.24.2
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.2.keytab
--principal=kudu/127.8.24.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:46527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 16:54:34.263819 32111 log.cc:824] T b5c8291ee75346bfbd3a66ae1077d5fb P e2efec6d440c4e2391890da0bd7e3a92: Log is configured to *not* fsync() on all Append() calls
I20260808 16:54:34.266475 32111 tablet_bootstrap.cc:492] T b5c8291ee75346bfbd3a66ae1077d5fb P e2efec6d440c4e2391890da0bd7e3a92: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 16:54:34.266776 32111 tablet_bootstrap.cc:492] T b5c8291ee75346bfbd3a66ae1077d5fb P e2efec6d440c4e2391890da0bd7e3a92: Bootstrap complete.
I20260808 16:54:34.267827 32111 ts_tablet_manager.cc:1397] T b5c8291ee75346bfbd3a66ae1077d5fb P e2efec6d440c4e2391890da0bd7e3a92: Time spent bootstrapping tablet: real 0.014s user 0.004s sys 0.008s
I20260808 16:54:34.269169 32111 raft_consensus.cc:348] T b5c8291ee75346bfbd3a66ae1077d5fb P e2efec6d440c4e2391890da0bd7e3a92 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "396866376cb04432abf9245607b453c2" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 37411 } } peers { permanent_uuid: "57fe315458094d32bff19b928d7f3e28" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 37085 } } peers { permanent_uuid: "e2efec6d440c4e2391890da0bd7e3a92" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 39989 } }
I20260808 16:54:34.269505 32111 raft_consensus.cc:729] T b5c8291ee75346bfbd3a66ae1077d5fb P e2efec6d440c4e2391890da0bd7e3a92 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2efec6d440c4e2391890da0bd7e3a92, State: Initialized, Role: FOLLOWER
I20260808 16:54:34.269647 32111 consensus_queue.cc:260] T b5c8291ee75346bfbd3a66ae1077d5fb P e2efec6d440c4e2391890da0bd7e3a92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "396866376cb04432abf9245607b453c2" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 37411 } } peers { permanent_uuid: "57fe315458094d32bff19b928d7f3e28" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 37085 } } peers { permanent_uuid: "e2efec6d440c4e2391890da0bd7e3a92" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 39989 } }
I20260808 16:54:34.269927 32111 ts_tablet_manager.cc:1428] T b5c8291ee75346bfbd3a66ae1077d5fb P e2efec6d440c4e2391890da0bd7e3a92: Time spent starting tablet: real 0.002s user 0.003s sys 0.001s
W20260808 16:54:34.342190 32273 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:54:34.342389 32273 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:54:34.342422 32273 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:54:34.343519 32273 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:54:34.343559 32273 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:54:34.343591 32273 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.2
I20260808 16:54:34.344996 32273 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:46527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.2.keytab
--principal=kudu/127.8.24.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.2:37085
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.24.2
--webserver_port=45533
--tserver_master_addrs=127.8.24.62:35859
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32273
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.2
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:54:34.345220 32273 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:54:34.345412 32273 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:54:34.349931 32287 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:54:34.349927 32285 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:54:34.350019 32291 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:54:34.350186 32273 server_base.cc:1034] running on GCE node
I20260808 16:54:34.350399 32273 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:54:34.350623 32273 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:54:34.354882 32273 hybrid_clock.cc:648] HybridClock initialized: now 1786208074354861 us; error 31 us; skew 500 ppm
Aug 08 16:54:34 dist-test-slave-6zbq krb5kdc[30116](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208074, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260808 16:54:34.357400 32276 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:54:34.264609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:35859 (local address 127.8.24.1:32887)
0808 16:54:34.264768 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:34.264816 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 16:54:34.265398 (+ 582us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:54:34.357192 (+ 91794us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:35859: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":82,"threads_started":1}
W20260808 16:54:34.357601 32270 heartbeater.cc:644] Failed to heartbeat to 127.8.24.62:35859 (0 consecutive failures): Network error: Failed to ping master at 127.8.24.62:35859: Client connection negotiation failed: client connection to 127.8.24.62:35859: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 16:54:34.357647 32273 init.cc:377] Logged in from keytab as kudu/127.8.24.2@KRBTEST.COM (short username kudu)
I20260808 16:54:34.357919 32276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:34.357801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:35859 (local address 127.8.24.1:56417)
0808 16:54:34.357846 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:34.357892 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260808 16:54:34.358122 32273 webserver.cc:460] Webserver started at http://127.8.24.2:45533/ using document root <none> and password file <none>
I20260808 16:54:34.358250 32276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:34.358154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:35859 (local address 127.8.24.1:56099)
0808 16:54:34.358203 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:34.358230 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260808 16:54:34.358387 32273 fs_manager.cc:359] Metadata directory not provided
W20260808 16:54:34.358402 32270 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 16:54:34.358453 32273 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:54:34.359665 32273 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 16:54:34.360134 32300 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:54:34.360273 32273 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 16:54:34.360356 32273 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-1/wal
uuid: "57fe315458094d32bff19b928d7f3e28"
format_stamp: "Formatted at 2026-08-08 16:54:32 on dist-test-slave-6zbq"
server_key: "2778c3b10f02cc8146e641346f5e74ab"
server_key_iv: "1a2dd2c36087f0ba9f98d3afcd71d2bf"
server_key_version: "encryptionkey@0"
I20260808 16:54:34.360639 32273 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:54:34.378955 32273 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:54:34.380510 32273 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:54:34.380609 32273 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:54:34.380760 32273 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:54:34.380990 32273 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:54:34.381018 32273 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:54:34.381037 32273 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:54:34.381053 32273 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:54:34.386384 32273 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.2:37085
I20260808 16:54:34.386442 32487 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.2:37085 every 8 connection(s)
I20260808 16:54:34.386723 32273 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-1/data/info.pb
I20260808 16:54:34.387636 32490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:34.387320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:35859 (local address 127.8.24.2:34643)
0808 16:54:34.387459 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:34.387517 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":69,"threads_started":1}
W20260808 16:54:34.387970 32488 heartbeater.cc:644] Failed to heartbeat to 127.8.24.62:35859 (0 consecutive failures): Network error: Failed to ping master at 127.8.24.62:35859: Client connection negotiation failed: client connection to 127.8.24.62:35859: connect: Connection refused (error 111)
I20260808 16:54:34.388202 32490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:34.388122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:35859 (local address 127.8.24.2:52705)
0808 16:54:34.388161 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:34.388185 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260808 16:54:34.388204 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 32273
I20260808 16:54:34.388429 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.3:37411
--local_ip_for_outbound_sockets=127.8.24.3
--tserver_master_addrs=127.8.24.62:35859
--webserver_port=44891
--webserver_interface=127.8.24.3
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.3.keytab
--principal=kudu/127.8.24.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:46527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 16:54:34.388504 32490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:34.388427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:35859 (local address 127.8.24.2:36677)
0808 16:54:34.388462 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:34.388484 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260808 16:54:34.388582 32488 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 16:54:34.466252 32491 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:54:34.466445 32491 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:54:34.466478 32491 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:54:34.467557 32491 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:54:34.467594 32491 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:54:34.467628 32491 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.3
I20260808 16:54:34.469291 32491 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:46527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.3.keytab
--principal=kudu/127.8.24.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.3:37411
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.24.3
--webserver_port=44891
--tserver_master_addrs=127.8.24.62:35859
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32491
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.3
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:54:34.469486 32491 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:54:34.469651 32491 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:54:34.474073 32505 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:54:34.474095 32508 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:54:34.474139 32500 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:54:34.474153 32491 server_base.cc:1034] running on GCE node
I20260808 16:54:34.474385 32491 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:54:34.474593 32491 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:54:34.475721 32491 hybrid_clock.cc:648] HybridClock initialized: now 1786208074475707 us; error 29 us; skew 500 ppm
Aug 08 16:54:34 dist-test-slave-6zbq krb5kdc[30116](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208074, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:54:34.477936 32491 init.cc:377] Logged in from keytab as kudu/127.8.24.3@KRBTEST.COM (short username kudu)
I20260808 16:54:34.478356 32491 webserver.cc:460] Webserver started at http://127.8.24.3:44891/ using document root <none> and password file <none>
I20260808 16:54:34.478547 32491 fs_manager.cc:359] Metadata directory not provided
I20260808 16:54:34.478593 32491 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:54:34.479732 32491 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 16:54:34.480201 32515 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:54:34.480345 32491 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 16:54:34.480423 32491 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-2/wal
uuid: "396866376cb04432abf9245607b453c2"
format_stamp: "Formatted at 2026-08-08 16:54:32 on dist-test-slave-6zbq"
server_key: "3345e3d822c4b152c4f06343904934b9"
server_key_iv: "ad80ce4eda7303fb9501cc055e140da7"
server_key_version: "encryptionkey@0"
I20260808 16:54:34.480711 32491 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:54:34.491907 32491 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:54:34.493772 32491 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:54:34.493891 32491 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:54:34.494074 32491 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:54:34.494328 32491 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:54:34.494390 32491 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:54:34.494426 32491 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:54:34.494450 32491 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:54:34.499763 32491 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.3:37411
I20260808 16:54:34.499799 32699 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.3:37411 every 8 connection(s)
I20260808 16:54:34.500080 32491 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786207919384338-8288-0/minicluster-data/ts-2/data/info.pb
I20260808 16:54:34.500970 32702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:34.500654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:35859 (local address 127.8.24.3:47713)
0808 16:54:34.500789 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:34.500848 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":65,"threads_started":1}
W20260808 16:54:34.501434 32700 heartbeater.cc:644] Failed to heartbeat to 127.8.24.62:35859 (0 consecutive failures): Network error: Failed to ping master at 127.8.24.62:35859: Client connection negotiation failed: client connection to 127.8.24.62:35859: connect: Connection refused (error 111)
I20260808 16:54:34.501711 32702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:34.501626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:35859 (local address 127.8.24.3:40217)
0808 16:54:34.501664 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:34.501691 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260808 16:54:34.501906 32702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:34.501861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:35859 (local address 127.8.24.3:41667)
0808 16:54:34.501878 (+ 17us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:34.501896 (+ 18us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":10}
W20260808 16:54:34.501986 32700 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 16:54:34.502830 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 32491
I20260808 16:54:35.358851 32706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:35.358661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:35859 (local address 127.8.24.1:46125)
0808 16:54:35.358765 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:35.358797 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":57,"threads_started":1}
I20260808 16:54:35.389027 32708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:35.388846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:35859 (local address 127.8.24.2:44145)
0808 16:54:35.388949 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:35.388979 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":57,"threads_started":1}
I20260808 16:54:35.502466 32710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:35.502268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:35859 (local address 127.8.24.3:58111)
0808 16:54:35.502391 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:35.502421 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":54,"threads_started":1}
I20260808 16:54:36.097834 32712 raft_consensus.cc:482] T b5c8291ee75346bfbd3a66ae1077d5fb P e2efec6d440c4e2391890da0bd7e3a92 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 16:54:36.097954 32712 raft_consensus.cc:504] T b5c8291ee75346bfbd3a66ae1077d5fb P e2efec6d440c4e2391890da0bd7e3a92 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "396866376cb04432abf9245607b453c2" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 37411 } } peers { permanent_uuid: "57fe315458094d32bff19b928d7f3e28" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 37085 } } peers { permanent_uuid: "e2efec6d440c4e2391890da0bd7e3a92" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 39989 } }
I20260808 16:54:36.098290 32712 leader_election.cc:288] T b5c8291ee75346bfbd3a66ae1077d5fb P e2efec6d440c4e2391890da0bd7e3a92 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 396866376cb04432abf9245607b453c2 (127.8.24.3:37411), 57fe315458094d32bff19b928d7f3e28 (127.8.24.2:37085)
Aug 08 16:54:36 dist-test-slave-6zbq krb5kdc[30116](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208074, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.1@KRBTEST.COM for kudu/127.8.24.2@KRBTEST.COM
Aug 08 16:54:36 dist-test-slave-6zbq krb5kdc[30116](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208074, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.1@KRBTEST.COM for kudu/127.8.24.3@KRBTEST.COM
I20260808 16:54:36.108171 32714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:36.098442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.3:37411 (local address 127.8.24.1:38651)
0808 16:54:36.098558 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:36.098570 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 16:54:36.098629 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:54:36.099611 (+ 982us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:54:36.099614 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:54:36.099959 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:54:36.100153 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:54:36.100159 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:54:36.101226 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:54:36.101227 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:54:36.101972 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:54:36.101974 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:54:36.102565 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:54:36.102567 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:54:36.102676 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:36.103194 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:54:36.103202 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:54:36.104984 (+ 1782us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:54:36.106887 (+ 1903us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:54:36.106888 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:54:36.106888 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:54:36.107028 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:54:36.107253 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:54:36.107253 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:54:36.107254 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:54:36.107306 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:54:36.107679 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:54:36.107680 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:54:36.107801 (+ 121us) client_negotiation.cc:769] Sending connection context
0808 16:54:36.107915 (+ 114us) client_negotiation.cc:240] Negotiation successful
0808 16:54:36.108071 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"mutex_wait_us":57,"thread_start_us":80,"threads_started":1}
I20260808 16:54:36.108227 32717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:36.098566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:38651 (local address 127.8.24.3:37411)
0808 16:54:36.098741 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0808 16:54:36.098753 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 16:54:36.098777 (+ 24us) server_negotiation.cc:407] Connection header received
0808 16:54:36.099344 (+ 567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:54:36.099347 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:54:36.099403 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:54:36.099550 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:54:36.100272 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:36.101132 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:36.102054 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:36.102484 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:36.102581 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:36.105075 (+ 2494us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:54:36.105079 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:54:36.105079 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:54:36.105111 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:54:36.106811 (+ 1700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:54:36.107106 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:54:36.107109 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:54:36.107109 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:54:36.107193 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:54:36.107358 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:54:36.107359 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:54:36.107359 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:54:36.107621 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:54:36.107691 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 16:54:36.107977 (+ 286us) server_negotiation.cc:299] Negotiation successful
0808 16:54:36.108130 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":70,"threads_started":1}
I20260808 16:54:36.108220 32715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:36.098556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:40743 (local address 127.8.24.2:37085)
0808 16:54:36.098741 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0808 16:54:36.098753 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 16:54:36.098776 (+ 23us) server_negotiation.cc:407] Connection header received
0808 16:54:36.099344 (+ 568us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:54:36.099347 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:54:36.099403 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:54:36.099550 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:54:36.100275 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:36.101118 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:36.102057 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:36.102484 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:36.102590 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:36.104656 (+ 2066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:54:36.104659 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:54:36.104660 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:54:36.104668 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:54:36.106543 (+ 1875us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:54:36.106902 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:54:36.106923 (+ 21us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:54:36.106923 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:54:36.107022 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:54:36.107249 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:54:36.107250 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:54:36.107251 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:54:36.107569 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:54:36.107666 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 16:54:36.107977 (+ 311us) server_negotiation.cc:299] Negotiation successful
0808 16:54:36.108111 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":64,"threads_started":1}
I20260808 16:54:36.108251 32713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:36.098443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.2:37085 (local address 127.8.24.1:40743)
0808 16:54:36.098585 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:36.098595 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 16:54:36.098650 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:54:36.099676 (+ 1026us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:54:36.099677 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:54:36.099960 (+ 283us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:54:36.100153 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:54:36.100159 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:54:36.101213 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:54:36.101214 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:54:36.101974 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:54:36.101976 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:54:36.102565 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:54:36.102566 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:54:36.102677 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:36.103244 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:54:36.103249 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:54:36.104549 (+ 1300us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:54:36.106613 (+ 2064us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:54:36.106620 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:54:36.106622 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:54:36.106821 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:54:36.107086 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:54:36.107086 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:54:36.107087 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:54:36.107188 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:54:36.107642 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:54:36.107649 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:54:36.107801 (+ 152us) client_negotiation.cc:769] Sending connection context
0808 16:54:36.107915 (+ 114us) client_negotiation.cc:240] Negotiation successful
0808 16:54:36.108153 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"mutex_wait_us":113,"thread_start_us":47,"threads_started":1}
W20260808 16:54:36.109460 32105 leader_election.cc:341] T b5c8291ee75346bfbd3a66ae1077d5fb P e2efec6d440c4e2391890da0bd7e3a92 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 396866376cb04432abf9245607b453c2 (127.8.24.3:37411): Not found: Tablet not found: b5c8291ee75346bfbd3a66ae1077d5fb
W20260808 16:54:36.109553 32104 leader_election.cc:341] T b5c8291ee75346bfbd3a66ae1077d5fb P e2efec6d440c4e2391890da0bd7e3a92 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 57fe315458094d32bff19b928d7f3e28 (127.8.24.2:37085): Not found: Tablet not found: b5c8291ee75346bfbd3a66ae1077d5fb
I20260808 16:54:36.109604 32104 leader_election.cc:302] T b5c8291ee75346bfbd3a66ae1077d5fb P e2efec6d440c4e2391890da0bd7e3a92 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e2efec6d440c4e2391890da0bd7e3a92; no voters: 396866376cb04432abf9245607b453c2, 57fe315458094d32bff19b928d7f3e28
I20260808 16:54:36.109696 32712 raft_consensus.cc:2726] T b5c8291ee75346bfbd3a66ae1077d5fb P e2efec6d440c4e2391890da0bd7e3a92 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 16:54:36.359409 32713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:36.359271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:35859 (local address 127.8.24.1:52647)
0808 16:54:36.359334 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:36.359368 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260808 16:54:36.389658 32718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:36.389420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:35859 (local address 127.8.24.2:49161)
0808 16:54:36.389550 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:36.389601 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":57,"threads_started":1}
I20260808 16:54:36.503065 32720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:36.502856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:35859 (local address 127.8.24.3:38207)
0808 16:54:36.502984 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:36.503026 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":73,"threads_started":1}
I20260808 16:54:37.360009 32721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:37.359828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:35859 (local address 127.8.24.1:60351)
0808 16:54:37.359935 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:37.359966 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":60,"threads_started":1}
I20260808 16:54:37.390200 32722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:37.390029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:35859 (local address 127.8.24.2:56269)
0808 16:54:37.390130 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:37.390163 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86,"thread_start_us":53,"threads_started":1}
I20260808 16:54:37.503605 32723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:37.503431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:35859 (local address 127.8.24.3:40327)
0808 16:54:37.503531 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:37.503564 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86,"thread_start_us":53,"threads_started":1}
I20260808 16:54:38.256314 32725 raft_consensus.cc:482] T b5c8291ee75346bfbd3a66ae1077d5fb P e2efec6d440c4e2391890da0bd7e3a92 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 16:54:38.256400 32725 raft_consensus.cc:504] T b5c8291ee75346bfbd3a66ae1077d5fb P e2efec6d440c4e2391890da0bd7e3a92 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "396866376cb04432abf9245607b453c2" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 37411 } } peers { permanent_uuid: "57fe315458094d32bff19b928d7f3e28" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 37085 } } peers { permanent_uuid: "e2efec6d440c4e2391890da0bd7e3a92" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 39989 } }
I20260808 16:54:38.256563 32725 leader_election.cc:288] T b5c8291ee75346bfbd3a66ae1077d5fb P e2efec6d440c4e2391890da0bd7e3a92 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 396866376cb04432abf9245607b453c2 (127.8.24.3:37411), 57fe315458094d32bff19b928d7f3e28 (127.8.24.2:37085)
W20260808 16:54:38.256942 32104 leader_election.cc:341] T b5c8291ee75346bfbd3a66ae1077d5fb P e2efec6d440c4e2391890da0bd7e3a92 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 57fe315458094d32bff19b928d7f3e28 (127.8.24.2:37085): Not found: Tablet not found: b5c8291ee75346bfbd3a66ae1077d5fb
W20260808 16:54:38.257025 32105 leader_election.cc:341] T b5c8291ee75346bfbd3a66ae1077d5fb P e2efec6d440c4e2391890da0bd7e3a92 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 396866376cb04432abf9245607b453c2 (127.8.24.3:37411): Not found: Tablet not found: b5c8291ee75346bfbd3a66ae1077d5fb
I20260808 16:54:38.257082 32105 leader_election.cc:302] T b5c8291ee75346bfbd3a66ae1077d5fb P e2efec6d440c4e2391890da0bd7e3a92 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e2efec6d440c4e2391890da0bd7e3a92; no voters: 396866376cb04432abf9245607b453c2, 57fe315458094d32bff19b928d7f3e28
I20260808 16:54:38.257160 32725 raft_consensus.cc:2726] T b5c8291ee75346bfbd3a66ae1077d5fb P e2efec6d440c4e2391890da0bd7e3a92 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 16:54:38.360656 32726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:38.360422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:35859 (local address 127.8.24.1:55971)
0808 16:54:38.360552 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:38.360589 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":78,"threads_started":1}
I20260808 16:54:38.390734 32727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:38.390533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:35859 (local address 127.8.24.2:52757)
0808 16:54:38.390657 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:38.390687 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":74,"threads_started":1}
I20260808 16:54:38.504169 32728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:38.503976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:35859 (local address 127.8.24.3:43241)
0808 16:54:38.504096 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:38.504125 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":75,"threads_started":1}
I20260808 16:54:39.361268 32730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:39.361044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:35859 (local address 127.8.24.1:59151)
0808 16:54:39.361188 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:39.361222 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":86,"threads_started":1}
I20260808 16:54:39.391312 32731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:39.391125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:35859 (local address 127.8.24.2:35479)
0808 16:54:39.391244 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:39.391276 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":73,"threads_started":1}
I20260808 16:54:39.504724 32734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:39.504527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:35859 (local address 127.8.24.3:36149)
0808 16:54:39.504654 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:39.504688 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":69,"threads_started":1}
I20260808 16:54:40.238417 32735 raft_consensus.cc:482] T b5c8291ee75346bfbd3a66ae1077d5fb P e2efec6d440c4e2391890da0bd7e3a92 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 16:54:40.238511 32735 raft_consensus.cc:504] T b5c8291ee75346bfbd3a66ae1077d5fb P e2efec6d440c4e2391890da0bd7e3a92 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "396866376cb04432abf9245607b453c2" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 37411 } } peers { permanent_uuid: "57fe315458094d32bff19b928d7f3e28" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 37085 } } peers { permanent_uuid: "e2efec6d440c4e2391890da0bd7e3a92" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 39989 } }
I20260808 16:54:40.238675 32735 leader_election.cc:288] T b5c8291ee75346bfbd3a66ae1077d5fb P e2efec6d440c4e2391890da0bd7e3a92 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 396866376cb04432abf9245607b453c2 (127.8.24.3:37411), 57fe315458094d32bff19b928d7f3e28 (127.8.24.2:37085)
W20260808 16:54:40.238998 32105 leader_election.cc:341] T b5c8291ee75346bfbd3a66ae1077d5fb P e2efec6d440c4e2391890da0bd7e3a92 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 396866376cb04432abf9245607b453c2 (127.8.24.3:37411): Not found: Tablet not found: b5c8291ee75346bfbd3a66ae1077d5fb
W20260808 16:54:40.239077 32104 leader_election.cc:341] T b5c8291ee75346bfbd3a66ae1077d5fb P e2efec6d440c4e2391890da0bd7e3a92 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 57fe315458094d32bff19b928d7f3e28 (127.8.24.2:37085): Not found: Tablet not found: b5c8291ee75346bfbd3a66ae1077d5fb
I20260808 16:54:40.239135 32104 leader_election.cc:302] T b5c8291ee75346bfbd3a66ae1077d5fb P e2efec6d440c4e2391890da0bd7e3a92 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e2efec6d440c4e2391890da0bd7e3a92; no voters: 396866376cb04432abf9245607b453c2, 57fe315458094d32bff19b928d7f3e28
I20260808 16:54:40.239208 32735 raft_consensus.cc:2726] T b5c8291ee75346bfbd3a66ae1077d5fb P e2efec6d440c4e2391890da0bd7e3a92 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 16:54:40.361804 32737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:40.361627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:35859 (local address 127.8.24.1:38409)
0808 16:54:40.361733 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:40.361760 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":58,"threads_started":1}
I20260808 16:54:40.391850 32738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:40.391652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:35859 (local address 127.8.24.2:56071)
0808 16:54:40.391767 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:40.391799 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":71,"threads_started":1}
I20260808 16:54:40.505285 32739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:40.505077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:35859 (local address 127.8.24.3:39459)
0808 16:54:40.505216 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:40.505247 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":71,"threads_started":1}
I20260808 16:54:41.362390 32741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:41.362160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:35859 (local address 127.8.24.1:41885)
0808 16:54:41.362285 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:41.362317 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":59,"threads_started":1}
I20260808 16:54:41.392326 32742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:41.392165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:35859 (local address 127.8.24.2:57009)
0808 16:54:41.392265 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:41.392293 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86,"thread_start_us":50,"threads_started":1}
I20260808 16:54:41.505865 32743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:41.505664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:35859 (local address 127.8.24.3:60669)
0808 16:54:41.505796 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:41.505829 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":71,"threads_started":1}
I20260808 16:54:42.363051 32745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:42.362854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:35859 (local address 127.8.24.1:56151)
0808 16:54:42.362969 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:42.363004 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":65,"threads_started":1}
I20260808 16:54:42.392879 32746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:42.392693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:35859 (local address 127.8.24.2:50525)
0808 16:54:42.392813 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:42.392842 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":57,"threads_started":1}
I20260808 16:54:42.506418 32749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:42.506215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:35859 (local address 127.8.24.3:54741)
0808 16:54:42.506320 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:42.506380 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":56,"threads_started":1}
I20260808 16:54:43.363595 32750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:43.363413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:35859 (local address 127.8.24.1:57981)
0808 16:54:43.363519 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:43.363550 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":60,"threads_started":1}
I20260808 16:54:43.393396 32751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:43.393214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:35859 (local address 127.8.24.2:60907)
0808 16:54:43.393337 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:43.393359 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":72,"threads_started":1}
I20260808 16:54:43.506961 32752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:43.506751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:35859 (local address 127.8.24.3:44577)
0808 16:54:43.506866 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:43.506915 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":70,"threads_started":1}
I20260808 16:54:44.364152 32753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:44.363967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:35859 (local address 127.8.24.1:33615)
0808 16:54:44.364072 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:44.364108 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":57,"threads_started":1}
I20260808 16:54:44.393920 32754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:44.393746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:35859 (local address 127.8.24.2:57711)
0808 16:54:44.393855 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:44.393883 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":61,"threads_started":1}
I20260808 16:54:44.446224 32755 raft_consensus.cc:482] T b5c8291ee75346bfbd3a66ae1077d5fb P e2efec6d440c4e2391890da0bd7e3a92 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 16:54:44.446329 32755 raft_consensus.cc:504] T b5c8291ee75346bfbd3a66ae1077d5fb P e2efec6d440c4e2391890da0bd7e3a92 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "396866376cb04432abf9245607b453c2" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 37411 } } peers { permanent_uuid: "57fe315458094d32bff19b928d7f3e28" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 37085 } } peers { permanent_uuid: "e2efec6d440c4e2391890da0bd7e3a92" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 39989 } }
I20260808 16:54:44.446501 32755 leader_election.cc:288] T b5c8291ee75346bfbd3a66ae1077d5fb P e2efec6d440c4e2391890da0bd7e3a92 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 396866376cb04432abf9245607b453c2 (127.8.24.3:37411), 57fe315458094d32bff19b928d7f3e28 (127.8.24.2:37085)
W20260808 16:54:44.446830 32104 leader_election.cc:341] T b5c8291ee75346bfbd3a66ae1077d5fb P e2efec6d440c4e2391890da0bd7e3a92 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 57fe315458094d32bff19b928d7f3e28 (127.8.24.2:37085): Not found: Tablet not found: b5c8291ee75346bfbd3a66ae1077d5fb
W20260808 16:54:44.446905 32105 leader_election.cc:341] T b5c8291ee75346bfbd3a66ae1077d5fb P e2efec6d440c4e2391890da0bd7e3a92 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 396866376cb04432abf9245607b453c2 (127.8.24.3:37411): Not found: Tablet not found: b5c8291ee75346bfbd3a66ae1077d5fb
I20260808 16:54:44.446949 32105 leader_election.cc:302] T b5c8291ee75346bfbd3a66ae1077d5fb P e2efec6d440c4e2391890da0bd7e3a92 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e2efec6d440c4e2391890da0bd7e3a92; no voters: 396866376cb04432abf9245607b453c2, 57fe315458094d32bff19b928d7f3e28
I20260808 16:54:44.447026 32755 raft_consensus.cc:2726] T b5c8291ee75346bfbd3a66ae1077d5fb P e2efec6d440c4e2391890da0bd7e3a92 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 16:54:44.507519 32756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:44.507315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:35859 (local address 127.8.24.3:35607)
0808 16:54:44.507446 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:44.507480 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":70,"threads_started":1}
I20260808 16:54:45.364856 32757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:45.364628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:35859 (local address 127.8.24.1:44591)
0808 16:54:45.364771 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:45.364802 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":79,"threads_started":1}
I20260808 16:54:45.394532 32758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:45.394359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:35859 (local address 127.8.24.2:40849)
0808 16:54:45.394467 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:45.394495 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":56,"threads_started":1}
I20260808 16:54:45.508026 32759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:45.507855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:35859 (local address 127.8.24.3:46193)
0808 16:54:45.507957 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:45.507989 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":57,"threads_started":1}
I20260808 16:54:46.365415 32761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:46.365229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:35859 (local address 127.8.24.1:52529)
0808 16:54:46.365339 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:46.365370 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":65,"threads_started":1}
I20260808 16:54:46.395045 32762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:46.394881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:35859 (local address 127.8.24.2:56983)
0808 16:54:46.394978 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:46.395009 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84,"thread_start_us":56,"threads_started":1}
I20260808 16:54:46.508539 32763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:46.508354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:35859 (local address 127.8.24.3:56645)
0808 16:54:46.508468 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:46.508502 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":72,"threads_started":1}
I20260808 16:54:47.365981 32764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:47.365757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:35859 (local address 127.8.24.1:60961)
0808 16:54:47.365898 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:47.365933 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":62,"threads_started":1}
I20260808 16:54:47.395533 32765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:47.395373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:35859 (local address 127.8.24.2:48443)
0808 16:54:47.395469 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:47.395500 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82,"thread_start_us":50,"threads_started":1}
I20260808 16:54:47.509052 32766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:47.508869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:35859 (local address 127.8.24.3:42537)
0808 16:54:47.508983 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:47.509014 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":69,"threads_started":1}
I20260808 16:54:48.184934 300 raft_consensus.cc:482] T b5c8291ee75346bfbd3a66ae1077d5fb P e2efec6d440c4e2391890da0bd7e3a92 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 16:54:48.185025 300 raft_consensus.cc:504] T b5c8291ee75346bfbd3a66ae1077d5fb P e2efec6d440c4e2391890da0bd7e3a92 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "396866376cb04432abf9245607b453c2" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 37411 } } peers { permanent_uuid: "57fe315458094d32bff19b928d7f3e28" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 37085 } } peers { permanent_uuid: "e2efec6d440c4e2391890da0bd7e3a92" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 39989 } }
I20260808 16:54:48.185169 300 leader_election.cc:288] T b5c8291ee75346bfbd3a66ae1077d5fb P e2efec6d440c4e2391890da0bd7e3a92 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 396866376cb04432abf9245607b453c2 (127.8.24.3:37411), 57fe315458094d32bff19b928d7f3e28 (127.8.24.2:37085)
W20260808 16:54:48.185494 32105 leader_election.cc:341] T b5c8291ee75346bfbd3a66ae1077d5fb P e2efec6d440c4e2391890da0bd7e3a92 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 396866376cb04432abf9245607b453c2 (127.8.24.3:37411): Not found: Tablet not found: b5c8291ee75346bfbd3a66ae1077d5fb
W20260808 16:54:48.185572 32104 leader_election.cc:341] T b5c8291ee75346bfbd3a66ae1077d5fb P e2efec6d440c4e2391890da0bd7e3a92 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 57fe315458094d32bff19b928d7f3e28 (127.8.24.2:37085): Not found: Tablet not found: b5c8291ee75346bfbd3a66ae1077d5fb
I20260808 16:54:48.185614 32104 leader_election.cc:302] T b5c8291ee75346bfbd3a66ae1077d5fb P e2efec6d440c4e2391890da0bd7e3a92 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e2efec6d440c4e2391890da0bd7e3a92; no voters: 396866376cb04432abf9245607b453c2, 57fe315458094d32bff19b928d7f3e28
I20260808 16:54:48.185679 300 raft_consensus.cc:2726] T b5c8291ee75346bfbd3a66ae1077d5fb P e2efec6d440c4e2391890da0bd7e3a92 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 16:54:48.366580 301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:48.366391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:35859 (local address 127.8.24.1:54629)
0808 16:54:48.366503 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:48.366534 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":58,"threads_started":1}
I20260808 16:54:48.396078 302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:48.395872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:35859 (local address 127.8.24.2:56455)
0808 16:54:48.395994 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:48.396026 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":75,"threads_started":1}
I20260808 16:54:48.509562 303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:48.509366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:35859 (local address 127.8.24.3:50685)
0808 16:54:48.509494 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:48.509526 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":68,"threads_started":1}
I20260808 16:54:49.367260 305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:49.367026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:35859 (local address 127.8.24.1:48177)
0808 16:54:49.367180 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:49.367209 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":102,"threads_started":1}
I20260808 16:54:49.396602 306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:49.396432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:35859 (local address 127.8.24.2:44439)
0808 16:54:49.396534 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:49.396565 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86,"thread_start_us":50,"threads_started":1}
I20260808 16:54:49.503623 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 32029
I20260808 16:54:49.510165 310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:49.509986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:35859 (local address 127.8.24.3:35083)
0808 16:54:49.510095 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:49.510123 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:35859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":49,"threads_started":1}
I20260808 16:54:49.516486 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 32273
I20260808 16:54:49.524060 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 32491
2026-08-08T16:54:49Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18355 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 16:54:49 dist-test-slave-6zbq krb5kdc[316](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 16:54:49 dist-test-slave-6zbq krb5kdc[316](info): set up 2 sockets
Aug 08 16:54:49 dist-test-slave-6zbq krb5kdc[316](info): commencing operation
krb5kdc: starting...
W20260808 16:54:50.661386 8288 mini_kdc.cc:121] Time spent starting KDC: real 1.110s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/krb5kdc/test-user.keytab.
Aug 08 16:54:50 dist-test-slave-6zbq krb5kdc[316](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208090, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T16:54:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T16:54:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.24.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
I20260808 16:54:50.766131 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:36933
--webserver_interface=127.8.24.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:36953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:36933
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:54:50.843552 337 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:54:50.843734 337 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:54:50.843770 337 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:54:50.844854 337 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 16:54:50.844892 337 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 16:54:50.844908 337 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 16:54:50.844923 337 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 16:54:50.844939 337 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:54:50.844964 337 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:54:50.844983 337 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 16:54:50.845006 337 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 16:54:50.846426 337 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:36953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:36933
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:36933
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.24.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.337
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:54:50.846654 337 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:54:50.846849 337 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 16:54:50.851435 337 server_base.cc:1034] running on GCE node
W20260808 16:54:50.851588 355 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:54:50.851488 357 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:54:50.851502 349 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:54:50.852083 337 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:54:50.852360 337 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:54:50.853497 337 hybrid_clock.cc:648] HybridClock initialized: now 1786208090853478 us; error 30 us; skew 500 ppm
Aug 08 16:54:50 dist-test-slave-6zbq krb5kdc[316](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208090, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:54:50.855728 337 init.cc:377] Logged in from keytab as kudu/127.8.24.62@KRBTEST.COM (short username kudu)
I20260808 16:54:50.856106 337 webserver.cc:460] Webserver started at http://127.8.24.62:41471/ using document root <none> and password file <none>
I20260808 16:54:50.856303 337 fs_manager.cc:359] Metadata directory not provided
I20260808 16:54:50.856351 337 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:54:50.856472 337 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:54:50.857417 337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/master-0/data/instance:
uuid: "a623e9fb44f445dcb72293c81486a593"
format_stamp: "Formatted at 2026-08-08 16:54:50 on dist-test-slave-6zbq"
server_key: "f60301cb1fbbc542e658a7993d7d3aff"
server_key_iv: "9df75af9e4ade91284cc9b04c23adfa0"
server_key_version: "encryptionkey@0"
I20260808 16:54:50.857728 337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/master-0/wal/instance:
uuid: "a623e9fb44f445dcb72293c81486a593"
format_stamp: "Formatted at 2026-08-08 16:54:50 on dist-test-slave-6zbq"
server_key: "f60301cb1fbbc542e658a7993d7d3aff"
server_key_iv: "9df75af9e4ade91284cc9b04c23adfa0"
server_key_version: "encryptionkey@0"
I20260808 16:54:50.858968 337 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 16:54:50.859603 368 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:54:50.859751 337 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 16:54:50.859834 337 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/master-0/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/master-0/wal
uuid: "a623e9fb44f445dcb72293c81486a593"
format_stamp: "Formatted at 2026-08-08 16:54:50 on dist-test-slave-6zbq"
server_key: "f60301cb1fbbc542e658a7993d7d3aff"
server_key_iv: "9df75af9e4ade91284cc9b04c23adfa0"
server_key_version: "encryptionkey@0"
I20260808 16:54:50.859894 337 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:54:50.874190 337 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:54:50.875725 337 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:54:50.875849 337 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:54:50.879421 337 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.62:36933
I20260808 16:54:50.879451 453 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.62:36933 every 8 connection(s)
I20260808 16:54:50.879760 337 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
I20260808 16:54:50.880524 454 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:54:50.882596 454 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a623e9fb44f445dcb72293c81486a593: Bootstrap starting.
I20260808 16:54:50.883128 454 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a623e9fb44f445dcb72293c81486a593: Neither blocks nor log segments found. Creating new log.
I20260808 16:54:50.883379 454 log.cc:824] T 00000000000000000000000000000000 P a623e9fb44f445dcb72293c81486a593: Log is configured to *not* fsync() on all Append() calls
I20260808 16:54:50.883975 454 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a623e9fb44f445dcb72293c81486a593: No bootstrap required, opened a new log
I20260808 16:54:50.885440 454 raft_consensus.cc:348] T 00000000000000000000000000000000 P a623e9fb44f445dcb72293c81486a593 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a623e9fb44f445dcb72293c81486a593" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 36933 } }
I20260808 16:54:50.885563 454 raft_consensus.cc:374] T 00000000000000000000000000000000 P a623e9fb44f445dcb72293c81486a593 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:54:50.885597 454 raft_consensus.cc:729] T 00000000000000000000000000000000 P a623e9fb44f445dcb72293c81486a593 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a623e9fb44f445dcb72293c81486a593, State: Initialized, Role: FOLLOWER
I20260808 16:54:50.885725 454 consensus_queue.cc:260] T 00000000000000000000000000000000 P a623e9fb44f445dcb72293c81486a593 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a623e9fb44f445dcb72293c81486a593" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 36933 } }
I20260808 16:54:50.885792 454 raft_consensus.cc:388] T 00000000000000000000000000000000 P a623e9fb44f445dcb72293c81486a593 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 16:54:50.885813 454 raft_consensus.cc:482] T 00000000000000000000000000000000 P a623e9fb44f445dcb72293c81486a593 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 16:54:50.885856 454 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a623e9fb44f445dcb72293c81486a593 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:54:50.886309 454 raft_consensus.cc:504] T 00000000000000000000000000000000 P a623e9fb44f445dcb72293c81486a593 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a623e9fb44f445dcb72293c81486a593" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 36933 } }
I20260808 16:54:50.886420 454 leader_election.cc:302] T 00000000000000000000000000000000 P a623e9fb44f445dcb72293c81486a593 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a623e9fb44f445dcb72293c81486a593; no voters:
I20260808 16:54:50.886567 454 leader_election.cc:288] T 00000000000000000000000000000000 P a623e9fb44f445dcb72293c81486a593 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 16:54:50.886605 461 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a623e9fb44f445dcb72293c81486a593 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:54:50.886742 461 raft_consensus.cc:686] T 00000000000000000000000000000000 P a623e9fb44f445dcb72293c81486a593 [term 1 LEADER]: Becoming Leader. State: Replica: a623e9fb44f445dcb72293c81486a593, State: Running, Role: LEADER
I20260808 16:54:50.886804 454 sys_catalog.cc:564] T 00000000000000000000000000000000 P a623e9fb44f445dcb72293c81486a593 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 16:54:50.886831 461 consensus_queue.cc:237] T 00000000000000000000000000000000 P a623e9fb44f445dcb72293c81486a593 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a623e9fb44f445dcb72293c81486a593" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 36933 } }
I20260808 16:54:50.887465 462 sys_catalog.cc:455] T 00000000000000000000000000000000 P a623e9fb44f445dcb72293c81486a593 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a623e9fb44f445dcb72293c81486a593" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a623e9fb44f445dcb72293c81486a593" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 36933 } } }
I20260808 16:54:50.887477 463 sys_catalog.cc:455] T 00000000000000000000000000000000 P a623e9fb44f445dcb72293c81486a593 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a623e9fb44f445dcb72293c81486a593. Latest consensus state: current_term: 1 leader_uuid: "a623e9fb44f445dcb72293c81486a593" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a623e9fb44f445dcb72293c81486a593" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 36933 } } }
I20260808 16:54:50.887569 462 sys_catalog.cc:458] T 00000000000000000000000000000000 P a623e9fb44f445dcb72293c81486a593 [sys.catalog]: This master's current role is: LEADER
I20260808 16:54:50.887578 463 sys_catalog.cc:458] T 00000000000000000000000000000000 P a623e9fb44f445dcb72293c81486a593 [sys.catalog]: This master's current role is: LEADER
I20260808 16:54:50.887787 473 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 16:54:50.888331 473 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 16:54:50.889712 473 catalog_manager.cc:1347] Generated new cluster ID: 92639f886e584e419174aa35f5ca4863
I20260808 16:54:50.889767 473 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 16:54:50.889910 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 337
I20260808 16:54:50.889994 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/master-0/wal/instance
I20260808 16:54:50.890184 8288 external_mini_cluster.cc:1428] Setting key dc292be13591ef68cc728db3175710d5
Aug 08 16:54:50 dist-test-slave-6zbq krb5kdc[316](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208090, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260808 16:54:50.898708 485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:50.890580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46922 (local address 127.8.24.62:36933)
0808 16:54:50.890738 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 16:54:50.890747 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 16:54:50.890780 (+ 33us) server_negotiation.cc:407] Connection header received
0808 16:54:50.890908 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:54:50.890914 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:54:50.890961 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:54:50.891070 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:54:50.891661 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:50.892554 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:50.893294 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:50.893694 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:50.893787 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:50.895883 (+ 2096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:54:50.895887 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:54:50.895888 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:54:50.895899 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:54:50.897327 (+ 1428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:54:50.897636 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:54:50.897639 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:54:50.897639 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:54:50.897726 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:54:50.897899 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:54:50.897900 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:54:50.897901 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:54:50.898123 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:54:50.898254 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0808 16:54:50.898384 (+ 130us) server_negotiation.cc:299] Negotiation successful
0808 16:54:50.898517 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":67,"threads_started":1}
I20260808 16:54:50.918888 473 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 16:54:50.919565 473 catalog_manager.cc:1504] Loading token signing keys...
I20260808 16:54:50.924974 473 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a623e9fb44f445dcb72293c81486a593: Generated new TSK 0
I20260808 16:54:50.925122 473 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.24.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.1.keytab.
Entry for principal kudu/127.8.24.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.1.keytab.
I20260808 16:54:50.950452 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.1:0
--local_ip_for_outbound_sockets=127.8.24.1
--webserver_interface=127.8.24.1
--webserver_port=0
--tserver_master_addrs=127.8.24.62:36933
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.1.keytab
--principal=kudu/127.8.24.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:36953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:54:51.029606 489 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:54:51.029783 489 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:54:51.029803 489 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:54:51.030877 489 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:54:51.030910 489 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:54:51.030934 489 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.1
I20260808 16:54:51.032308 489 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:36953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.1.keytab
--principal=kudu/127.8.24.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.24.1
--webserver_port=0
--tserver_master_addrs=127.8.24.62:36933
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.489
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.1
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:54:51.032478 489 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:54:51.032639 489 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:54:51.037026 500 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:54:51.037034 503 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:54:51.037048 489 server_base.cc:1034] running on GCE node
W20260808 16:54:51.037045 507 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:54:51.037457 489 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:54:51.037685 489 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:54:51.038813 489 hybrid_clock.cc:648] HybridClock initialized: now 1786208091038799 us; error 27 us; skew 500 ppm
Aug 08 16:54:51 dist-test-slave-6zbq krb5kdc[316](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208091, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:54:51.041133 489 init.cc:377] Logged in from keytab as kudu/127.8.24.1@KRBTEST.COM (short username kudu)
I20260808 16:54:51.041493 489 webserver.cc:460] Webserver started at http://127.8.24.1:33579/ using document root <none> and password file <none>
I20260808 16:54:51.041672 489 fs_manager.cc:359] Metadata directory not provided
I20260808 16:54:51.041710 489 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:54:51.041795 489 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:54:51.042724 489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-0/data/instance:
uuid: "1d47cc8d97044f9aa2943ce89c4efaaa"
format_stamp: "Formatted at 2026-08-08 16:54:51 on dist-test-slave-6zbq"
server_key: "f2d1cdd77f20479ac7b19045512bb25d"
server_key_iv: "703017eb4f461fbc6dfd65dac530115c"
server_key_version: "encryptionkey@0"
I20260808 16:54:51.043004 489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-0/wal/instance:
uuid: "1d47cc8d97044f9aa2943ce89c4efaaa"
format_stamp: "Formatted at 2026-08-08 16:54:51 on dist-test-slave-6zbq"
server_key: "f2d1cdd77f20479ac7b19045512bb25d"
server_key_iv: "703017eb4f461fbc6dfd65dac530115c"
server_key_version: "encryptionkey@0"
I20260808 16:54:51.044123 489 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 16:54:51.044727 515 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:54:51.044878 489 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 16:54:51.044950 489 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-0/wal
uuid: "1d47cc8d97044f9aa2943ce89c4efaaa"
format_stamp: "Formatted at 2026-08-08 16:54:51 on dist-test-slave-6zbq"
server_key: "f2d1cdd77f20479ac7b19045512bb25d"
server_key_iv: "703017eb4f461fbc6dfd65dac530115c"
server_key_version: "encryptionkey@0"
I20260808 16:54:51.045011 489 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:54:51.057711 489 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:54:51.059233 489 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:54:51.059353 489 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:54:51.059526 489 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:54:51.059782 489 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:54:51.059821 489 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:54:51.059840 489 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:54:51.059862 489 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:54:51.065013 489 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.1:37163
I20260808 16:54:51.065073 695 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.1:37163 every 8 connection(s)
I20260808 16:54:51.065316 489 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-0/data/info.pb
Aug 08 16:54:51 dist-test-slave-6zbq krb5kdc[316](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208091, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.1@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260808 16:54:51.073025 485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:51.065873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:43627 (local address 127.8.24.62:36933)
0808 16:54:51.065913 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 16:54:51.065914 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 16:54:51.066103 (+ 189us) server_negotiation.cc:407] Connection header received
0808 16:54:51.066546 (+ 443us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:54:51.066547 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:54:51.066557 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:54:51.066592 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:54:51.067393 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:51.067865 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:51.068727 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:51.068912 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:51.068952 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:51.070805 (+ 1853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:54:51.070807 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:54:51.070807 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:54:51.070814 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:54:51.071862 (+ 1048us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:54:51.072199 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:54:51.072200 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:54:51.072201 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:54:51.072255 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:54:51.072460 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:54:51.072461 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:54:51.072461 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:54:51.072563 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:54:51.072617 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 16:54:51.072858 (+ 241us) server_negotiation.cc:299] Negotiation successful
0808 16:54:51.072938 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 16:54:51.073151 699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:51.065858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:36933 (local address 127.8.24.1:43627)
0808 16:54:51.065989 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:51.066019 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0808 16:54:51.066456 (+ 437us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:54:51.066657 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:54:51.066660 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:54:51.067157 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:54:51.067329 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:54:51.067333 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:54:51.067922 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:54:51.067922 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:54:51.068669 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:54:51.068671 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:54:51.068962 (+ 291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:54:51.068962 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:54:51.069051 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:51.069452 (+ 401us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:54:51.069460 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:54:51.070719 (+ 1259us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:54:51.071934 (+ 1215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:54:51.071941 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:54:51.071943 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:54:51.072135 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:54:51.072310 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:54:51.072311 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:54:51.072311 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:54:51.072406 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:54:51.072616 (+ 210us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:54:51.072622 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:54:51.072743 (+ 121us) client_negotiation.cc:769] Sending connection context
0808 16:54:51.072829 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 16:54:51.072967 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":77,"threads_started":1}
I20260808 16:54:51.073559 697 heartbeater.cc:347] Connected to a master server at 127.8.24.62:36933
I20260808 16:54:51.073645 697 heartbeater.cc:464] Registering TS with master...
I20260808 16:54:51.073804 697 heartbeater.cc:511] Master 127.8.24.62:36933 requested a full tablet report, sending...
I20260808 16:54:51.074270 397 ts_manager.cc:194] Registered new tserver with Master: 1d47cc8d97044f9aa2943ce89c4efaaa (127.8.24.1:37163)
I20260808 16:54:51.074553 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 489
I20260808 16:54:51.074666 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-0/wal/instance
I20260808 16:54:51.074872 8288 external_mini_cluster.cc:1428] Setting key d8fbe7fd550a6db0ed9bba6f7b019877
I20260808 16:54:51.075103 397 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.1@KRBTEST.COM'} at 127.8.24.1:43627
WARNING: no policy specified for kudu/127.8.24.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.2.keytab.
Entry for principal kudu/127.8.24.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.2.keytab.
I20260808 16:54:51.096942 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.2:0
--local_ip_for_outbound_sockets=127.8.24.2
--webserver_interface=127.8.24.2
--webserver_port=0
--tserver_master_addrs=127.8.24.62:36933
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.2.keytab
--principal=kudu/127.8.24.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:36953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:54:51.175444 710 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:54:51.175625 710 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:54:51.175647 710 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:54:51.176730 710 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:54:51.176769 710 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:54:51.176792 710 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.2
I20260808 16:54:51.178200 710 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:36953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.2.keytab
--principal=kudu/127.8.24.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.24.2
--webserver_port=0
--tserver_master_addrs=127.8.24.62:36933
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.710
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.2
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:54:51.178413 710 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:54:51.178617 710 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:54:51.183139 721 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:54:51.183159 719 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:54:51.183331 710 server_base.cc:1034] running on GCE node
W20260808 16:54:51.183156 724 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:54:51.183553 710 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:54:51.183776 710 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:54:51.184912 710 hybrid_clock.cc:648] HybridClock initialized: now 1786208091184902 us; error 26 us; skew 500 ppm
Aug 08 16:54:51 dist-test-slave-6zbq krb5kdc[316](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208091, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:54:51.187285 710 init.cc:377] Logged in from keytab as kudu/127.8.24.2@KRBTEST.COM (short username kudu)
I20260808 16:54:51.187711 710 webserver.cc:460] Webserver started at http://127.8.24.2:45671/ using document root <none> and password file <none>
I20260808 16:54:51.187947 710 fs_manager.cc:359] Metadata directory not provided
I20260808 16:54:51.187991 710 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:54:51.188112 710 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:54:51.189077 710 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-1/data/instance:
uuid: "275153aa50f44d19b906434156c28e31"
format_stamp: "Formatted at 2026-08-08 16:54:51 on dist-test-slave-6zbq"
server_key: "daa72f3463c2e38c9867de56b3e7cd98"
server_key_iv: "c7125c51b2b17114d39058e2334f00d9"
server_key_version: "encryptionkey@0"
I20260808 16:54:51.189402 710 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-1/wal/instance:
uuid: "275153aa50f44d19b906434156c28e31"
format_stamp: "Formatted at 2026-08-08 16:54:51 on dist-test-slave-6zbq"
server_key: "daa72f3463c2e38c9867de56b3e7cd98"
server_key_iv: "c7125c51b2b17114d39058e2334f00d9"
server_key_version: "encryptionkey@0"
I20260808 16:54:51.190621 710 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 16:54:51.191267 734 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:54:51.191404 710 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 16:54:51.191480 710 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-1/wal
uuid: "275153aa50f44d19b906434156c28e31"
format_stamp: "Formatted at 2026-08-08 16:54:51 on dist-test-slave-6zbq"
server_key: "daa72f3463c2e38c9867de56b3e7cd98"
server_key_iv: "c7125c51b2b17114d39058e2334f00d9"
server_key_version: "encryptionkey@0"
I20260808 16:54:51.191543 710 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:54:51.219815 710 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:54:51.221529 710 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:54:51.221655 710 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:54:51.221866 710 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:54:51.222139 710 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:54:51.222172 710 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:54:51.222193 710 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:54:51.222214 710 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:54:51.227487 710 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.2:44295
I20260808 16:54:51.227537 898 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.2:44295 every 8 connection(s)
I20260808 16:54:51.227807 710 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-1/data/info.pb
I20260808 16:54:51.231114 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 710
I20260808 16:54:51.231209 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-1/wal/instance
I20260808 16:54:51.231384 8288 external_mini_cluster.cc:1428] Setting key f08d051e49e8c9a6b24df47c99cde7b2
Aug 08 16:54:51 dist-test-slave-6zbq krb5kdc[316](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208091, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.2@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260808 16:54:51.236431 485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:51.228409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:44041 (local address 127.8.24.62:36933)
0808 16:54:51.228453 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 16:54:51.228454 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 16:54:51.228638 (+ 184us) server_negotiation.cc:407] Connection header received
0808 16:54:51.229089 (+ 451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:54:51.229089 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:54:51.229098 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:54:51.229152 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:54:51.229992 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:51.230490 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:51.231478 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:51.231722 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:51.231787 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:51.233877 (+ 2090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:54:51.233880 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:54:51.233880 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:54:51.233887 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:54:51.235240 (+ 1353us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:54:51.235589 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:54:51.235591 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:54:51.235591 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:54:51.235655 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:54:51.235847 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:54:51.235848 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:54:51.235848 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:54:51.235972 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:54:51.236033 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 16:54:51.236253 (+ 220us) server_negotiation.cc:299] Negotiation successful
0808 16:54:51.236347 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 16:54:51.236455 903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:51.228378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:36933 (local address 127.8.24.2:44041)
0808 16:54:51.228528 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:51.228567 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 16:54:51.229014 (+ 447us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:54:51.229211 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:54:51.229214 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:54:51.229748 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:54:51.229928 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:54:51.229933 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:54:51.230547 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:54:51.230547 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:54:51.231386 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:54:51.231388 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:54:51.231793 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:54:51.231794 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:54:51.231904 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:51.232355 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:54:51.232363 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:54:51.233768 (+ 1405us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:54:51.235314 (+ 1546us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:54:51.235320 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:54:51.235321 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:54:51.235519 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:54:51.235712 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:54:51.235713 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:54:51.235713 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:54:51.235797 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:54:51.236025 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:54:51.236028 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:54:51.236125 (+ 97us) client_negotiation.cc:769] Sending connection context
0808 16:54:51.236200 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 16:54:51.236310 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":69,"threads_started":1}
I20260808 16:54:51.236944 900 heartbeater.cc:347] Connected to a master server at 127.8.24.62:36933
I20260808 16:54:51.237023 900 heartbeater.cc:464] Registering TS with master...
I20260808 16:54:51.237177 900 heartbeater.cc:511] Master 127.8.24.62:36933 requested a full tablet report, sending...
I20260808 16:54:51.237548 396 ts_manager.cc:194] Registered new tserver with Master: 275153aa50f44d19b906434156c28e31 (127.8.24.2:44295)
I20260808 16:54:51.238027 396 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.2@KRBTEST.COM'} at 127.8.24.2:44041
WARNING: no policy specified for kudu/127.8.24.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.3.keytab.
Entry for principal kudu/127.8.24.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.3.keytab.
I20260808 16:54:51.254565 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.3:0
--local_ip_for_outbound_sockets=127.8.24.3
--webserver_interface=127.8.24.3
--webserver_port=0
--tserver_master_addrs=127.8.24.62:36933
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.3.keytab
--principal=kudu/127.8.24.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:36953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:54:51.331791 909 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:54:51.331971 909 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:54:51.331992 909 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:54:51.333046 909 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:54:51.333081 909 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:54:51.333106 909 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.3
I20260808 16:54:51.334503 909 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:36953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.3.keytab
--principal=kudu/127.8.24.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.24.3
--webserver_port=0
--tserver_master_addrs=127.8.24.62:36933
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.909
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.3
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:54:51.334666 909 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:54:51.334828 909 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:54:51.339388 918 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:54:51.339340 921 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:54:51.339345 917 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:54:51.339416 909 server_base.cc:1034] running on GCE node
I20260808 16:54:51.339722 909 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:54:51.339939 909 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:54:51.341068 909 hybrid_clock.cc:648] HybridClock initialized: now 1786208091341059 us; error 26 us; skew 500 ppm
Aug 08 16:54:51 dist-test-slave-6zbq krb5kdc[316](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208091, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:54:51.343367 909 init.cc:377] Logged in from keytab as kudu/127.8.24.3@KRBTEST.COM (short username kudu)
I20260808 16:54:51.343770 909 webserver.cc:460] Webserver started at http://127.8.24.3:39907/ using document root <none> and password file <none>
I20260808 16:54:51.343933 909 fs_manager.cc:359] Metadata directory not provided
I20260808 16:54:51.343969 909 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:54:51.344053 909 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:54:51.344952 909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-2/data/instance:
uuid: "d99276ea3dad479c9a077324c1e7dccc"
format_stamp: "Formatted at 2026-08-08 16:54:51 on dist-test-slave-6zbq"
server_key: "4f814eb022fcb67fa8f59f0e700aca0b"
server_key_iv: "4f681b16dacc81905593ce21969b24df"
server_key_version: "encryptionkey@0"
I20260808 16:54:51.345237 909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-2/wal/instance:
uuid: "d99276ea3dad479c9a077324c1e7dccc"
format_stamp: "Formatted at 2026-08-08 16:54:51 on dist-test-slave-6zbq"
server_key: "4f814eb022fcb67fa8f59f0e700aca0b"
server_key_iv: "4f681b16dacc81905593ce21969b24df"
server_key_version: "encryptionkey@0"
I20260808 16:54:51.346725 909 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 16:54:51.347398 929 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:54:51.347530 909 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 16:54:51.347604 909 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-2/wal
uuid: "d99276ea3dad479c9a077324c1e7dccc"
format_stamp: "Formatted at 2026-08-08 16:54:51 on dist-test-slave-6zbq"
server_key: "4f814eb022fcb67fa8f59f0e700aca0b"
server_key_iv: "4f681b16dacc81905593ce21969b24df"
server_key_version: "encryptionkey@0"
I20260808 16:54:51.347668 909 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:54:51.358036 909 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:54:51.359634 909 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:54:51.359748 909 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:54:51.359930 909 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:54:51.360205 909 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:54:51.360239 909 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:54:51.360260 909 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:54:51.360282 909 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:54:51.365651 909 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.3:33733
I20260808 16:54:51.365686 1094 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.3:33733 every 8 connection(s)
I20260808 16:54:51.365943 909 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-2/data/info.pb
I20260808 16:54:51.368247 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 909
I20260808 16:54:51.368345 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-2/wal/instance
I20260808 16:54:51.368515 8288 external_mini_cluster.cc:1428] Setting key 65ab649a08d69c5582dfb5245a20e021
Aug 08 16:54:51 dist-test-slave-6zbq krb5kdc[316](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208091, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.3@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260808 16:54:51.373785 485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:51.366554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:35705 (local address 127.8.24.62:36933)
0808 16:54:51.366607 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 16:54:51.366609 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:54:51.366799 (+ 190us) server_negotiation.cc:407] Connection header received
0808 16:54:51.367218 (+ 419us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:54:51.367219 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:54:51.367230 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:54:51.367268 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:54:51.368065 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:51.368599 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:51.369536 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:51.369723 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:51.369780 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:51.371628 (+ 1848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:54:51.371629 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:54:51.371629 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:54:51.371635 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:54:51.372831 (+ 1196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:54:51.373124 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:54:51.373124 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:54:51.373125 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:54:51.373168 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:54:51.373339 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:54:51.373340 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:54:51.373340 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:54:51.373429 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:54:51.373472 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0808 16:54:51.373661 (+ 189us) server_negotiation.cc:299] Negotiation successful
0808 16:54:51.373714 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 16:54:51.373939 1097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:51.366539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:36933 (local address 127.8.24.3:35705)
0808 16:54:51.366689 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:51.366721 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0808 16:54:51.367134 (+ 413us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:54:51.367321 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:54:51.367324 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:54:51.367828 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:54:51.367995 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:54:51.368000 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:54:51.368699 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:54:51.368700 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:54:51.369466 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:54:51.369468 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:54:51.369795 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:54:51.369796 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:54:51.369891 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:51.370267 (+ 376us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:54:51.370275 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:54:51.371540 (+ 1265us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:54:51.372889 (+ 1349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:54:51.372891 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:54:51.372893 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:54:51.373073 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:54:51.373212 (+ 139us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:54:51.373212 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:54:51.373212 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:54:51.373297 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:54:51.373471 (+ 174us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:54:51.373474 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:54:51.373572 (+ 98us) client_negotiation.cc:769] Sending connection context
0808 16:54:51.373644 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 16:54:51.373756 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":85,"threads_started":1}
I20260808 16:54:51.374384 1095 heartbeater.cc:347] Connected to a master server at 127.8.24.62:36933
I20260808 16:54:51.374468 1095 heartbeater.cc:464] Registering TS with master...
I20260808 16:54:51.374629 1095 heartbeater.cc:511] Master 127.8.24.62:36933 requested a full tablet report, sending...
I20260808 16:54:51.374950 396 ts_manager.cc:194] Registered new tserver with Master: d99276ea3dad479c9a077324c1e7dccc (127.8.24.3:33733)
I20260808 16:54:51.375296 396 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.3@KRBTEST.COM'} at 127.8.24.3:35705
I20260808 16:54:51.379976 8288 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 16:54:51.386312 485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:51.381216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46934 (local address 127.8.24.62:36933)
0808 16:54:51.381263 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 16:54:51.381265 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:54:51.381379 (+ 114us) server_negotiation.cc:407] Connection header received
0808 16:54:51.381437 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:54:51.381438 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:54:51.381447 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:54:51.381490 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:54:51.381999 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:51.382491 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:51.383189 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:51.383382 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:51.383436 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:51.384386 (+ 950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:54:51.384387 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:54:51.384388 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:54:51.384394 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:54:51.385339 (+ 945us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:54:51.385625 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:54:51.385626 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:54:51.385626 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:54:51.385675 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:54:51.385834 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:54:51.385834 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:54:51.385835 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:54:51.385961 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:54:51.386046 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 16:54:51.386146 (+ 100us) server_negotiation.cc:299] Negotiation successful
0808 16:54:51.386230 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 16:54:51.387621 396 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46934:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 16:54:51.389197 396 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 16:54:51.395109 1106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:51.391042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.2:44295 (local address 127.0.0.1:45526)
0808 16:54:51.391304 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:51.391313 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 16:54:51.391364 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:54:51.391670 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:54:51.391674 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:54:51.391677 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:54:51.391738 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:54:51.391740 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:54:51.393045 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:54:51.393047 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:54:51.393970 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:54:51.393972 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:54:51.394849 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:54:51.394850 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:54:51.394916 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:51.394919 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 16:54:51.394981 (+ 62us) client_negotiation.cc:240] Negotiation successful
0808 16:54:51.395024 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"mutex_wait_us":39,"thread_start_us":45,"threads_started":1}
I20260808 16:54:51.395110 1108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:51.391169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39130 (local address 127.8.24.3:33733)
0808 16:54:51.391742 (+ 573us) server_negotiation.cc:206] Beginning negotiation
0808 16:54:51.391746 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 16:54:51.391754 (+ 8us) server_negotiation.cc:407] Connection header received
0808 16:54:51.391781 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:54:51.391783 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:54:51.391796 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:54:51.391865 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:54:51.392031 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:51.392680 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:51.394018 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:51.394642 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:51.394703 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:51.394768 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 16:54:51.394996 (+ 228us) server_negotiation.cc:299] Negotiation successful
0808 16:54:51.395037 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":545,"thread_start_us":59,"threads_started":1}
I20260808 16:54:51.395292 1110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:51.391207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:37163 (local address 127.0.0.1:52116)
0808 16:54:51.391398 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:51.391407 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 16:54:51.391440 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:54:51.391751 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:54:51.391752 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:54:51.391765 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:54:51.391818 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:54:51.391820 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:54:51.393105 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:54:51.393106 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:54:51.394201 (+ 1095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:54:51.394204 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:54:51.395088 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:54:51.395089 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:54:51.395141 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:51.395144 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 16:54:51.395186 (+ 42us) client_negotiation.cc:240] Negotiation successful
0808 16:54:51.395221 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":56,"threads_started":1}
I20260808 16:54:51.395355 1107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:51.391144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45526 (local address 127.8.24.2:44295)
0808 16:54:51.391537 (+ 393us) server_negotiation.cc:206] Beginning negotiation
0808 16:54:51.391540 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 16:54:51.391548 (+ 8us) server_negotiation.cc:407] Connection header received
0808 16:54:51.391577 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:54:51.391579 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:54:51.391590 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:54:51.391646 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:54:51.391828 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:51.392927 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:51.394033 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:51.394732 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:51.394834 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:51.394888 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 16:54:51.395232 (+ 344us) server_negotiation.cc:299] Negotiation successful
0808 16:54:51.395280 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361,"thread_start_us":80,"threads_started":1}
I20260808 16:54:51.395383 1111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:51.391432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52116 (local address 127.8.24.1:37163)
0808 16:54:51.391579 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0808 16:54:51.391583 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 16:54:51.391593 (+ 10us) server_negotiation.cc:407] Connection header received
0808 16:54:51.391628 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:54:51.391630 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:54:51.391645 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:54:51.391739 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:54:51.391986 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:51.393012 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:51.394278 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:51.394992 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:51.395082 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:51.395249 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0808 16:54:51.395272 (+ 23us) server_negotiation.cc:299] Negotiation successful
0808 16:54:51.395312 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":70,"threads_started":1}
I20260808 16:54:51.395678 1103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:51.391042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.3:33733 (local address 127.0.0.1:39130)
0808 16:54:51.391181 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:51.391215 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0808 16:54:51.391273 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:54:51.391926 (+ 653us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:54:51.391927 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:54:51.391929 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:54:51.391959 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:54:51.391960 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:54:51.392778 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:54:51.392779 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:54:51.393929 (+ 1150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:54:51.393932 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:54:51.394803 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:54:51.394804 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:54:51.394883 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:51.394896 (+ 13us) client_negotiation.cc:769] Sending connection context
0808 16:54:51.394963 (+ 67us) client_negotiation.cc:240] Negotiation successful
0808 16:54:51.395024 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"mutex_wait_us":5,"thread_start_us":52,"threads_started":1}
I20260808 16:54:51.396039 596 tablet_service.cc:1467] Processing CreateTablet for tablet 2cd8f81ed87443b6b656616784da4846 (DEFAULT_TABLE table=test-table [id=e0a2f2f3fa184a028eebb7219bb5f13e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:54:51.396039 808 tablet_service.cc:1467] Processing CreateTablet for tablet 2cd8f81ed87443b6b656616784da4846 (DEFAULT_TABLE table=test-table [id=e0a2f2f3fa184a028eebb7219bb5f13e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:54:51.396232 1000 tablet_service.cc:1467] Processing CreateTablet for tablet 2cd8f81ed87443b6b656616784da4846 (DEFAULT_TABLE table=test-table [id=e0a2f2f3fa184a028eebb7219bb5f13e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:54:51.396379 596 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2cd8f81ed87443b6b656616784da4846. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:54:51.396378 808 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2cd8f81ed87443b6b656616784da4846. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:54:51.396471 1000 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2cd8f81ed87443b6b656616784da4846. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:54:51.398994 1112 tablet_bootstrap.cc:492] T 2cd8f81ed87443b6b656616784da4846 P d99276ea3dad479c9a077324c1e7dccc: Bootstrap starting.
I20260808 16:54:51.399534 1112 tablet_bootstrap.cc:654] T 2cd8f81ed87443b6b656616784da4846 P d99276ea3dad479c9a077324c1e7dccc: Neither blocks nor log segments found. Creating new log.
I20260808 16:54:51.399524 1113 tablet_bootstrap.cc:492] T 2cd8f81ed87443b6b656616784da4846 P 275153aa50f44d19b906434156c28e31: Bootstrap starting.
I20260808 16:54:51.399752 1114 tablet_bootstrap.cc:492] T 2cd8f81ed87443b6b656616784da4846 P 1d47cc8d97044f9aa2943ce89c4efaaa: Bootstrap starting.
I20260808 16:54:51.400105 1112 log.cc:824] T 2cd8f81ed87443b6b656616784da4846 P d99276ea3dad479c9a077324c1e7dccc: Log is configured to *not* fsync() on all Append() calls
I20260808 16:54:51.400151 1113 tablet_bootstrap.cc:654] T 2cd8f81ed87443b6b656616784da4846 P 275153aa50f44d19b906434156c28e31: Neither blocks nor log segments found. Creating new log.
I20260808 16:54:51.400309 1114 tablet_bootstrap.cc:654] T 2cd8f81ed87443b6b656616784da4846 P 1d47cc8d97044f9aa2943ce89c4efaaa: Neither blocks nor log segments found. Creating new log.
I20260808 16:54:51.400609 1113 log.cc:824] T 2cd8f81ed87443b6b656616784da4846 P 275153aa50f44d19b906434156c28e31: Log is configured to *not* fsync() on all Append() calls
I20260808 16:54:51.400895 1112 tablet_bootstrap.cc:492] T 2cd8f81ed87443b6b656616784da4846 P d99276ea3dad479c9a077324c1e7dccc: No bootstrap required, opened a new log
I20260808 16:54:51.400888 1114 log.cc:824] T 2cd8f81ed87443b6b656616784da4846 P 1d47cc8d97044f9aa2943ce89c4efaaa: Log is configured to *not* fsync() on all Append() calls
I20260808 16:54:51.400955 1112 ts_tablet_manager.cc:1397] T 2cd8f81ed87443b6b656616784da4846 P d99276ea3dad479c9a077324c1e7dccc: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 16:54:51.401319 1113 tablet_bootstrap.cc:492] T 2cd8f81ed87443b6b656616784da4846 P 275153aa50f44d19b906434156c28e31: No bootstrap required, opened a new log
I20260808 16:54:51.401437 1113 ts_tablet_manager.cc:1397] T 2cd8f81ed87443b6b656616784da4846 P 275153aa50f44d19b906434156c28e31: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 16:54:51.401631 1114 tablet_bootstrap.cc:492] T 2cd8f81ed87443b6b656616784da4846 P 1d47cc8d97044f9aa2943ce89c4efaaa: No bootstrap required, opened a new log
I20260808 16:54:51.401705 1114 ts_tablet_manager.cc:1397] T 2cd8f81ed87443b6b656616784da4846 P 1d47cc8d97044f9aa2943ce89c4efaaa: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 16:54:51.402899 1113 raft_consensus.cc:348] T 2cd8f81ed87443b6b656616784da4846 P 275153aa50f44d19b906434156c28e31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "275153aa50f44d19b906434156c28e31" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 44295 } } peers { permanent_uuid: "d99276ea3dad479c9a077324c1e7dccc" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 33733 } } peers { permanent_uuid: "1d47cc8d97044f9aa2943ce89c4efaaa" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 37163 } }
I20260808 16:54:51.402869 1112 raft_consensus.cc:348] T 2cd8f81ed87443b6b656616784da4846 P d99276ea3dad479c9a077324c1e7dccc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "275153aa50f44d19b906434156c28e31" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 44295 } } peers { permanent_uuid: "d99276ea3dad479c9a077324c1e7dccc" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 33733 } } peers { permanent_uuid: "1d47cc8d97044f9aa2943ce89c4efaaa" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 37163 } }
I20260808 16:54:51.403033 1112 raft_consensus.cc:374] T 2cd8f81ed87443b6b656616784da4846 P d99276ea3dad479c9a077324c1e7dccc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:54:51.403033 1113 raft_consensus.cc:374] T 2cd8f81ed87443b6b656616784da4846 P 275153aa50f44d19b906434156c28e31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:54:51.403066 1112 raft_consensus.cc:729] T 2cd8f81ed87443b6b656616784da4846 P d99276ea3dad479c9a077324c1e7dccc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d99276ea3dad479c9a077324c1e7dccc, State: Initialized, Role: FOLLOWER
I20260808 16:54:51.403074 1113 raft_consensus.cc:729] T 2cd8f81ed87443b6b656616784da4846 P 275153aa50f44d19b906434156c28e31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 275153aa50f44d19b906434156c28e31, State: Initialized, Role: FOLLOWER
I20260808 16:54:51.403204 1113 consensus_queue.cc:260] T 2cd8f81ed87443b6b656616784da4846 P 275153aa50f44d19b906434156c28e31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "275153aa50f44d19b906434156c28e31" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 44295 } } peers { permanent_uuid: "d99276ea3dad479c9a077324c1e7dccc" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 33733 } } peers { permanent_uuid: "1d47cc8d97044f9aa2943ce89c4efaaa" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 37163 } }
I20260808 16:54:51.403225 1112 consensus_queue.cc:260] T 2cd8f81ed87443b6b656616784da4846 P d99276ea3dad479c9a077324c1e7dccc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "275153aa50f44d19b906434156c28e31" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 44295 } } peers { permanent_uuid: "d99276ea3dad479c9a077324c1e7dccc" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 33733 } } peers { permanent_uuid: "1d47cc8d97044f9aa2943ce89c4efaaa" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 37163 } }
I20260808 16:54:51.403475 1113 ts_tablet_manager.cc:1428] T 2cd8f81ed87443b6b656616784da4846 P 275153aa50f44d19b906434156c28e31: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260808 16:54:51.403503 1112 ts_tablet_manager.cc:1428] T 2cd8f81ed87443b6b656616784da4846 P d99276ea3dad479c9a077324c1e7dccc: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 16:54:51.403475 900 heartbeater.cc:503] Master 127.8.24.62:36933 was elected leader, sending a full tablet report...
I20260808 16:54:51.403631 1095 heartbeater.cc:503] Master 127.8.24.62:36933 was elected leader, sending a full tablet report...
I20260808 16:54:51.403573 1114 raft_consensus.cc:348] T 2cd8f81ed87443b6b656616784da4846 P 1d47cc8d97044f9aa2943ce89c4efaaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "275153aa50f44d19b906434156c28e31" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 44295 } } peers { permanent_uuid: "d99276ea3dad479c9a077324c1e7dccc" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 33733 } } peers { permanent_uuid: "1d47cc8d97044f9aa2943ce89c4efaaa" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 37163 } }
I20260808 16:54:51.403716 1114 raft_consensus.cc:374] T 2cd8f81ed87443b6b656616784da4846 P 1d47cc8d97044f9aa2943ce89c4efaaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:54:51.403741 1114 raft_consensus.cc:729] T 2cd8f81ed87443b6b656616784da4846 P 1d47cc8d97044f9aa2943ce89c4efaaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d47cc8d97044f9aa2943ce89c4efaaa, State: Initialized, Role: FOLLOWER
I20260808 16:54:51.403827 1114 consensus_queue.cc:260] T 2cd8f81ed87443b6b656616784da4846 P 1d47cc8d97044f9aa2943ce89c4efaaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "275153aa50f44d19b906434156c28e31" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 44295 } } peers { permanent_uuid: "d99276ea3dad479c9a077324c1e7dccc" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 33733 } } peers { permanent_uuid: "1d47cc8d97044f9aa2943ce89c4efaaa" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 37163 } }
I20260808 16:54:51.404081 1114 ts_tablet_manager.cc:1428] T 2cd8f81ed87443b6b656616784da4846 P 1d47cc8d97044f9aa2943ce89c4efaaa: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 16:54:51.404112 697 heartbeater.cc:503] Master 127.8.24.62:36933 was elected leader, sending a full tablet report...
W20260808 16:54:51.478322 901 tablet.cc:2369] T 2cd8f81ed87443b6b656616784da4846 P 275153aa50f44d19b906434156c28e31: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 16:54:51.566013 698 tablet.cc:2369] T 2cd8f81ed87443b6b656616784da4846 P 1d47cc8d97044f9aa2943ce89c4efaaa: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 16:54:51.605500 1127 raft_consensus.cc:482] T 2cd8f81ed87443b6b656616784da4846 P 1d47cc8d97044f9aa2943ce89c4efaaa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 16:54:51.605599 1127 raft_consensus.cc:504] T 2cd8f81ed87443b6b656616784da4846 P 1d47cc8d97044f9aa2943ce89c4efaaa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "275153aa50f44d19b906434156c28e31" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 44295 } } peers { permanent_uuid: "d99276ea3dad479c9a077324c1e7dccc" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 33733 } } peers { permanent_uuid: "1d47cc8d97044f9aa2943ce89c4efaaa" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 37163 } }
I20260808 16:54:51.605950 1127 leader_election.cc:288] T 2cd8f81ed87443b6b656616784da4846 P 1d47cc8d97044f9aa2943ce89c4efaaa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 275153aa50f44d19b906434156c28e31 (127.8.24.2:44295), d99276ea3dad479c9a077324c1e7dccc (127.8.24.3:33733)
I20260808 16:54:51.609704 1107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:51.606136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:46723 (local address 127.8.24.2:44295)
0808 16:54:51.606179 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 16:54:51.606181 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:54:51.606301 (+ 120us) server_negotiation.cc:407] Connection header received
0808 16:54:51.606361 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:54:51.606362 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:54:51.606373 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:54:51.606431 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:54:51.606602 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:51.607336 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:51.608797 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:51.609366 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:51.609438 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:51.609449 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 16:54:51.609596 (+ 147us) server_negotiation.cc:299] Negotiation successful
0808 16:54:51.609633 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 16:54:51.609704 1108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:51.606210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:57923 (local address 127.8.24.3:33733)
0808 16:54:51.606255 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 16:54:51.606257 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:54:51.606270 (+ 13us) server_negotiation.cc:407] Connection header received
0808 16:54:51.606359 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:54:51.606360 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:54:51.606371 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:54:51.606431 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:54:51.606602 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:51.607349 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:51.608812 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:51.609376 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:51.609438 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:51.609449 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 16:54:51.609596 (+ 147us) server_negotiation.cc:299] Negotiation successful
0808 16:54:51.609632 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 16:54:51.609736 1128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:51.606099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.3:33733 (local address 127.8.24.1:57923)
0808 16:54:51.606224 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:51.606233 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 16:54:51.606284 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:54:51.606442 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:54:51.606442 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:54:51.606444 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:54:51.606537 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:54:51.606538 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:54:51.607410 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:54:51.607410 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:54:51.608721 (+ 1311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:54:51.608724 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:54:51.609444 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:54:51.609445 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:54:51.609529 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:51.609535 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 16:54:51.609585 (+ 50us) client_negotiation.cc:240] Negotiation successful
0808 16:54:51.609633 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"mutex_wait_us":59,"thread_start_us":79,"threads_started":1}
I20260808 16:54:51.609736 1130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:51.606081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.2:44295 (local address 127.8.24.1:46723)
0808 16:54:51.606219 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:51.606229 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 16:54:51.606293 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:54:51.606442 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:54:51.606443 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:54:51.606458 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:54:51.606535 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:54:51.606537 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:54:51.607408 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:54:51.607409 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:54:51.608721 (+ 1312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:54:51.608724 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:54:51.609446 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:54:51.609446 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:54:51.609529 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:51.609534 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 16:54:51.609585 (+ 51us) client_negotiation.cc:240] Negotiation successful
0808 16:54:51.609632 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"mutex_wait_us":93,"thread_start_us":77,"threads_started":1}
I20260808 16:54:51.610216 836 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cd8f81ed87443b6b656616784da4846" candidate_uuid: "1d47cc8d97044f9aa2943ce89c4efaaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "275153aa50f44d19b906434156c28e31" is_pre_election: true
I20260808 16:54:51.610303 1033 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cd8f81ed87443b6b656616784da4846" candidate_uuid: "1d47cc8d97044f9aa2943ce89c4efaaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d99276ea3dad479c9a077324c1e7dccc" is_pre_election: true
I20260808 16:54:51.610373 836 raft_consensus.cc:2454] T 2cd8f81ed87443b6b656616784da4846 P 275153aa50f44d19b906434156c28e31 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1d47cc8d97044f9aa2943ce89c4efaaa in term 0.
I20260808 16:54:51.610474 1033 raft_consensus.cc:2454] T 2cd8f81ed87443b6b656616784da4846 P d99276ea3dad479c9a077324c1e7dccc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1d47cc8d97044f9aa2943ce89c4efaaa in term 0.
I20260808 16:54:51.610595 521 leader_election.cc:302] T 2cd8f81ed87443b6b656616784da4846 P 1d47cc8d97044f9aa2943ce89c4efaaa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d47cc8d97044f9aa2943ce89c4efaaa, 275153aa50f44d19b906434156c28e31; no voters:
I20260808 16:54:51.610713 1127 raft_consensus.cc:2781] T 2cd8f81ed87443b6b656616784da4846 P 1d47cc8d97044f9aa2943ce89c4efaaa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 16:54:51.610770 1127 raft_consensus.cc:482] T 2cd8f81ed87443b6b656616784da4846 P 1d47cc8d97044f9aa2943ce89c4efaaa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 16:54:51.610792 1127 raft_consensus.cc:3037] T 2cd8f81ed87443b6b656616784da4846 P 1d47cc8d97044f9aa2943ce89c4efaaa [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:54:51.611471 1127 raft_consensus.cc:504] T 2cd8f81ed87443b6b656616784da4846 P 1d47cc8d97044f9aa2943ce89c4efaaa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "275153aa50f44d19b906434156c28e31" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 44295 } } peers { permanent_uuid: "d99276ea3dad479c9a077324c1e7dccc" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 33733 } } peers { permanent_uuid: "1d47cc8d97044f9aa2943ce89c4efaaa" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 37163 } }
I20260808 16:54:51.611590 1127 leader_election.cc:288] T 2cd8f81ed87443b6b656616784da4846 P 1d47cc8d97044f9aa2943ce89c4efaaa [CANDIDATE]: Term 1 election: Requested vote from peers 275153aa50f44d19b906434156c28e31 (127.8.24.2:44295), d99276ea3dad479c9a077324c1e7dccc (127.8.24.3:33733)
I20260808 16:54:51.611720 836 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cd8f81ed87443b6b656616784da4846" candidate_uuid: "1d47cc8d97044f9aa2943ce89c4efaaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "275153aa50f44d19b906434156c28e31"
I20260808 16:54:51.611747 1033 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cd8f81ed87443b6b656616784da4846" candidate_uuid: "1d47cc8d97044f9aa2943ce89c4efaaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d99276ea3dad479c9a077324c1e7dccc"
I20260808 16:54:51.611801 836 raft_consensus.cc:3037] T 2cd8f81ed87443b6b656616784da4846 P 275153aa50f44d19b906434156c28e31 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:54:51.611814 1033 raft_consensus.cc:3037] T 2cd8f81ed87443b6b656616784da4846 P d99276ea3dad479c9a077324c1e7dccc [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:54:51.612469 836 raft_consensus.cc:2454] T 2cd8f81ed87443b6b656616784da4846 P 275153aa50f44d19b906434156c28e31 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d47cc8d97044f9aa2943ce89c4efaaa in term 1.
I20260808 16:54:51.612470 1033 raft_consensus.cc:2454] T 2cd8f81ed87443b6b656616784da4846 P d99276ea3dad479c9a077324c1e7dccc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d47cc8d97044f9aa2943ce89c4efaaa in term 1.
I20260808 16:54:51.612630 521 leader_election.cc:302] T 2cd8f81ed87443b6b656616784da4846 P 1d47cc8d97044f9aa2943ce89c4efaaa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d47cc8d97044f9aa2943ce89c4efaaa, 275153aa50f44d19b906434156c28e31; no voters:
I20260808 16:54:51.612727 1127 raft_consensus.cc:2781] T 2cd8f81ed87443b6b656616784da4846 P 1d47cc8d97044f9aa2943ce89c4efaaa [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:54:51.612862 1127 raft_consensus.cc:686] T 2cd8f81ed87443b6b656616784da4846 P 1d47cc8d97044f9aa2943ce89c4efaaa [term 1 LEADER]: Becoming Leader. State: Replica: 1d47cc8d97044f9aa2943ce89c4efaaa, State: Running, Role: LEADER
I20260808 16:54:51.612980 1127 consensus_queue.cc:237] T 2cd8f81ed87443b6b656616784da4846 P 1d47cc8d97044f9aa2943ce89c4efaaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "275153aa50f44d19b906434156c28e31" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 44295 } } peers { permanent_uuid: "d99276ea3dad479c9a077324c1e7dccc" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 33733 } } peers { permanent_uuid: "1d47cc8d97044f9aa2943ce89c4efaaa" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 37163 } }
I20260808 16:54:51.613758 397 catalog_manager.cc:5495] T 2cd8f81ed87443b6b656616784da4846 P 1d47cc8d97044f9aa2943ce89c4efaaa reported cstate change: term changed from 0 to 1, leader changed from <none> to 1d47cc8d97044f9aa2943ce89c4efaaa (127.8.24.1). New cstate: current_term: 1 leader_uuid: "1d47cc8d97044f9aa2943ce89c4efaaa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "275153aa50f44d19b906434156c28e31" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 44295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d99276ea3dad479c9a077324c1e7dccc" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 33733 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1d47cc8d97044f9aa2943ce89c4efaaa" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 37163 } health_report { overall_health: HEALTHY } } }
W20260808 16:54:51.616420 1096 tablet.cc:2369] T 2cd8f81ed87443b6b656616784da4846 P d99276ea3dad479c9a077324c1e7dccc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 16:54:51.619928 1111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:51.617750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52124 (local address 127.8.24.1:37163)
0808 16:54:51.617798 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 16:54:51.617800 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:54:51.617809 (+ 9us) server_negotiation.cc:407] Connection header received
0808 16:54:51.617864 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:54:51.617865 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:54:51.617883 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:54:51.617937 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 16:54:51.618059 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:51.618539 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:51.619239 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:54:51.619438 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:54:51.619492 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:54:51.619643 (+ 151us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 16:54:51.619741 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 16:54:51.619808 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 16:54:51.619852 (+ 44us) server_negotiation.cc:299] Negotiation successful
0808 16:54:51.619869 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 16:54:51.621340 836 raft_consensus.cc:1261] T 2cd8f81ed87443b6b656616784da4846 P 275153aa50f44d19b906434156c28e31 [term 1 FOLLOWER]: Refusing update from remote peer 1d47cc8d97044f9aa2943ce89c4efaaa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:54:51.621361 1033 raft_consensus.cc:1261] T 2cd8f81ed87443b6b656616784da4846 P d99276ea3dad479c9a077324c1e7dccc [term 1 FOLLOWER]: Refusing update from remote peer 1d47cc8d97044f9aa2943ce89c4efaaa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:54:51.621610 1127 consensus_queue.cc:1035] T 2cd8f81ed87443b6b656616784da4846 P 1d47cc8d97044f9aa2943ce89c4efaaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "275153aa50f44d19b906434156c28e31" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:54:51.621743 1131 consensus_queue.cc:1035] T 2cd8f81ed87443b6b656616784da4846 P 1d47cc8d97044f9aa2943ce89c4efaaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "d99276ea3dad479c9a077324c1e7dccc" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 33733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:54:51.624625 1137 mvcc.cc:205] Tried to move back new op lower bound from 7316308343279046656 to 7316308343247417344. Current Snapshot: MvccSnapshot[applied={T|T < 7316308343279046656}]
I20260808 16:54:51.625715 396 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46934:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 16:54:51.625803 396 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46934:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 16:54:51.626457 396 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a623e9fb44f445dcb72293c81486a593: Sending DeleteTablet for 3 replicas of tablet 2cd8f81ed87443b6b656616784da4846
I20260808 16:54:51.626784 596 tablet_service.cc:1514] Processing DeleteTablet for tablet 2cd8f81ed87443b6b656616784da4846 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:54:51 UTC) from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:52116
I20260808 16:54:51.626781 808 tablet_service.cc:1514] Processing DeleteTablet for tablet 2cd8f81ed87443b6b656616784da4846 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:54:51 UTC) from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:45526
I20260808 16:54:51.626789 1000 tablet_service.cc:1514] Processing DeleteTablet for tablet 2cd8f81ed87443b6b656616784da4846 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:54:51 UTC) from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:39130
I20260808 16:54:51.627002 1150 tablet_replica.cc:330] T 2cd8f81ed87443b6b656616784da4846 P 275153aa50f44d19b906434156c28e31: stopping tablet replica
I20260808 16:54:51.627101 1150 raft_consensus.cc:2229] T 2cd8f81ed87443b6b656616784da4846 P 275153aa50f44d19b906434156c28e31 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 16:54:51.627097 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 489
I20260808 16:54:51.633509 1150 raft_consensus.cc:2258] T 2cd8f81ed87443b6b656616784da4846 P 275153aa50f44d19b906434156c28e31 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:54:51.634900 1150 ts_tablet_manager.cc:1905] T 2cd8f81ed87443b6b656616784da4846 P 275153aa50f44d19b906434156c28e31: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:54:51.636168 1150 ts_tablet_manager.cc:1918] T 2cd8f81ed87443b6b656616784da4846 P 275153aa50f44d19b906434156c28e31: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 16:54:51.636229 1150 log.cc:1196] T 2cd8f81ed87443b6b656616784da4846 P 275153aa50f44d19b906434156c28e31: Deleting WAL directory at /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-1/wal/wals/2cd8f81ed87443b6b656616784da4846
I20260808 16:54:51.636430 1150 ts_tablet_manager.cc:1939] T 2cd8f81ed87443b6b656616784da4846 P 275153aa50f44d19b906434156c28e31: Deleting consensus metadata
I20260808 16:54:51.636778 374 catalog_manager.cc:4841] TS 275153aa50f44d19b906434156c28e31 (127.8.24.2:44295): tablet 2cd8f81ed87443b6b656616784da4846 (table test-table [id=e0a2f2f3fa184a028eebb7219bb5f13e]) successfully deleted
I20260808 16:54:51.638505 1148 tablet_replica.cc:330] T 2cd8f81ed87443b6b656616784da4846 P d99276ea3dad479c9a077324c1e7dccc: stopping tablet replica
I20260808 16:54:51.638589 1148 raft_consensus.cc:2229] T 2cd8f81ed87443b6b656616784da4846 P d99276ea3dad479c9a077324c1e7dccc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 16:54:51.638684 1148 raft_consensus.cc:2258] T 2cd8f81ed87443b6b656616784da4846 P d99276ea3dad479c9a077324c1e7dccc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:54:51.639015 1148 ts_tablet_manager.cc:1905] T 2cd8f81ed87443b6b656616784da4846 P d99276ea3dad479c9a077324c1e7dccc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:54:51.640168 1148 ts_tablet_manager.cc:1918] T 2cd8f81ed87443b6b656616784da4846 P d99276ea3dad479c9a077324c1e7dccc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 16:54:51.640223 1148 log.cc:1196] T 2cd8f81ed87443b6b656616784da4846 P d99276ea3dad479c9a077324c1e7dccc: Deleting WAL directory at /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-2/wal/wals/2cd8f81ed87443b6b656616784da4846
I20260808 16:54:51.640460 1148 ts_tablet_manager.cc:1939] T 2cd8f81ed87443b6b656616784da4846 P d99276ea3dad479c9a077324c1e7dccc: Deleting consensus metadata
W20260808 16:54:51.640493 370 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.24.1:37163 (error 108)
I20260808 16:54:51.640727 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 710
I20260808 16:54:51.640847 370 catalog_manager.cc:4841] TS d99276ea3dad479c9a077324c1e7dccc (127.8.24.3:33733): tablet 2cd8f81ed87443b6b656616784da4846 (table test-table [id=e0a2f2f3fa184a028eebb7219bb5f13e]) successfully deleted
I20260808 16:54:51.644091 1103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:54:51.640812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:37163 (local address 127.0.0.1:52140)
0808 16:54:51.644023 (+ 3211us) negotiation.cc:107] Waiting for socket to connect
0808 16:54:51.644065 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.1:37163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3188}
W20260808 16:54:51.649698 370 catalog_manager.cc:4584] TS 1d47cc8d97044f9aa2943ce89c4efaaa (127.8.24.1:37163): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2cd8f81ed87443b6b656616784da4846: Network error: Client connection negotiation failed: client connection to 127.8.24.1:37163: connect: Connection refused (error 111)
I20260808 16:54:51.659998 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 909
I20260808 16:54:51.668049 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 337
I20260808 16:55:11.675704 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:36933
--webserver_interface=127.8.24.62
--webserver_port=41471
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:36953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:36933
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:55:11.757612 1153 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:55:11.757790 1153 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:55:11.757810 1153 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:55:11.758934 1153 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 16:55:11.758971 1153 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 16:55:11.758983 1153 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 16:55:11.759004 1153 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 16:55:11.759025 1153 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:55:11.759047 1153 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:55:11.759069 1153 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 16:55:11.759093 1153 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 16:55:11.760542 1153 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:36953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:36933
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:36933
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.24.62
--webserver_port=41471
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1153
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:55:11.760766 1153 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:55:11.760960 1153 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:55:11.768999 1161 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:55:11.769130 1153 server_base.cc:1034] running on GCE node
W20260808 16:55:11.769003 1165 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:55:11.769035 1162 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:55:11.769474 1153 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:55:11.769762 1153 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:55:11.770890 1153 hybrid_clock.cc:648] HybridClock initialized: now 1786208111770881 us; error 27 us; skew 500 ppm
Aug 08 16:55:11 dist-test-slave-6zbq krb5kdc[316](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208111, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:55:11.775902 1153 init.cc:377] Logged in from keytab as kudu/127.8.24.62@KRBTEST.COM (short username kudu)
I20260808 16:55:11.776314 1153 webserver.cc:460] Webserver started at http://127.8.24.62:41471/ using document root <none> and password file <none>
I20260808 16:55:11.776525 1153 fs_manager.cc:359] Metadata directory not provided
I20260808 16:55:11.776579 1153 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:55:11.777840 1153 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 16:55:11.778421 1173 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:55:11.778566 1153 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 16:55:11.778673 1153 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/master-0/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/master-0/wal
uuid: "a623e9fb44f445dcb72293c81486a593"
format_stamp: "Formatted at 2026-08-08 16:54:50 on dist-test-slave-6zbq"
server_key: "f60301cb1fbbc542e658a7993d7d3aff"
server_key_iv: "9df75af9e4ade91284cc9b04c23adfa0"
server_key_version: "encryptionkey@0"
I20260808 16:55:11.779021 1153 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:55:11.797001 1153 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:55:11.798635 1153 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:55:11.798753 1153 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:55:11.802594 1153 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.62:36933
I20260808 16:55:11.802646 1263 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.62:36933 every 8 connection(s)
I20260808 16:55:11.802986 1153 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
I20260808 16:55:11.804332 1264 sys_catalog.cc:263] Verifying existing consensus state
I20260808 16:55:11.804963 1264 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a623e9fb44f445dcb72293c81486a593: Bootstrap starting.
I20260808 16:55:11.811694 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 1153
I20260808 16:55:11.811925 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.1:37163
--local_ip_for_outbound_sockets=127.8.24.1
--tserver_master_addrs=127.8.24.62:36933
--webserver_port=33579
--webserver_interface=127.8.24.1
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.1.keytab
--principal=kudu/127.8.24.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:36953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 16:55:11.813226 1264 log.cc:824] T 00000000000000000000000000000000 P a623e9fb44f445dcb72293c81486a593: Log is configured to *not* fsync() on all Append() calls
I20260808 16:55:11.815032 1264 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a623e9fb44f445dcb72293c81486a593: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 16:55:11.815267 1264 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a623e9fb44f445dcb72293c81486a593: Bootstrap complete.
I20260808 16:55:11.817351 1264 raft_consensus.cc:348] T 00000000000000000000000000000000 P a623e9fb44f445dcb72293c81486a593 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a623e9fb44f445dcb72293c81486a593" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 36933 } }
I20260808 16:55:11.817570 1264 raft_consensus.cc:729] T 00000000000000000000000000000000 P a623e9fb44f445dcb72293c81486a593 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a623e9fb44f445dcb72293c81486a593, State: Initialized, Role: FOLLOWER
I20260808 16:55:11.817726 1264 consensus_queue.cc:260] T 00000000000000000000000000000000 P a623e9fb44f445dcb72293c81486a593 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a623e9fb44f445dcb72293c81486a593" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 36933 } }
I20260808 16:55:11.817786 1264 raft_consensus.cc:388] T 00000000000000000000000000000000 P a623e9fb44f445dcb72293c81486a593 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 16:55:11.817837 1264 raft_consensus.cc:482] T 00000000000000000000000000000000 P a623e9fb44f445dcb72293c81486a593 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 16:55:11.817883 1264 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a623e9fb44f445dcb72293c81486a593 [term 1 FOLLOWER]: Advancing to term 2
I20260808 16:55:11.818867 1264 raft_consensus.cc:504] T 00000000000000000000000000000000 P a623e9fb44f445dcb72293c81486a593 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a623e9fb44f445dcb72293c81486a593" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 36933 } }
I20260808 16:55:11.818964 1264 leader_election.cc:302] T 00000000000000000000000000000000 P a623e9fb44f445dcb72293c81486a593 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a623e9fb44f445dcb72293c81486a593; no voters:
I20260808 16:55:11.819114 1264 leader_election.cc:288] T 00000000000000000000000000000000 P a623e9fb44f445dcb72293c81486a593 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 16:55:11.819160 1271 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a623e9fb44f445dcb72293c81486a593 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 16:55:11.819326 1271 raft_consensus.cc:686] T 00000000000000000000000000000000 P a623e9fb44f445dcb72293c81486a593 [term 2 LEADER]: Becoming Leader. State: Replica: a623e9fb44f445dcb72293c81486a593, State: Running, Role: LEADER
I20260808 16:55:11.819372 1264 sys_catalog.cc:564] T 00000000000000000000000000000000 P a623e9fb44f445dcb72293c81486a593 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 16:55:11.819423 1271 consensus_queue.cc:237] T 00000000000000000000000000000000 P a623e9fb44f445dcb72293c81486a593 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a623e9fb44f445dcb72293c81486a593" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 36933 } }
I20260808 16:55:11.819705 1271 sys_catalog.cc:455] T 00000000000000000000000000000000 P a623e9fb44f445dcb72293c81486a593 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a623e9fb44f445dcb72293c81486a593. Latest consensus state: current_term: 2 leader_uuid: "a623e9fb44f445dcb72293c81486a593" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a623e9fb44f445dcb72293c81486a593" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 36933 } } }
I20260808 16:55:11.819941 1271 sys_catalog.cc:458] T 00000000000000000000000000000000 P a623e9fb44f445dcb72293c81486a593 [sys.catalog]: This master's current role is: LEADER
I20260808 16:55:11.819870 1274 sys_catalog.cc:455] T 00000000000000000000000000000000 P a623e9fb44f445dcb72293c81486a593 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "a623e9fb44f445dcb72293c81486a593" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a623e9fb44f445dcb72293c81486a593" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 36933 } } }
I20260808 16:55:11.820326 1274 sys_catalog.cc:458] T 00000000000000000000000000000000 P a623e9fb44f445dcb72293c81486a593 [sys.catalog]: This master's current role is: LEADER
I20260808 16:55:11.820430 1284 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 16:55:11.821295 1284 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 16:55:11.821398 1284 catalog_manager.cc:1259] Loaded cluster ID: 92639f886e584e419174aa35f5ca4863
I20260808 16:55:11.821431 1284 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 16:55:11.822607 1284 catalog_manager.cc:1504] Loading token signing keys...
I20260808 16:55:11.822789 1284 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a623e9fb44f445dcb72293c81486a593: Loaded TSK: 0
I20260808 16:55:11.831667 1284 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a623e9fb44f445dcb72293c81486a593: Generated new TSK 1
I20260808 16:55:11.831753 1284 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 16:55:11.892021 1269 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:55:11.892204 1269 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:55:11.892235 1269 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:55:11.893306 1269 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:55:11.893344 1269 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:55:11.893378 1269 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.1
I20260808 16:55:11.894793 1269 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:36953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.1.keytab
--principal=kudu/127.8.24.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.1:37163
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.24.1
--webserver_port=33579
--tserver_master_addrs=127.8.24.62:36933
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1269
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.1
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:55:11.895000 1269 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:55:11.895198 1269 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:55:11.900672 1304 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:55:11.900789 1269 server_base.cc:1034] running on GCE node
W20260808 16:55:11.900646 1305 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:55:11.900656 1307 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:55:11.901165 1269 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:55:11.901430 1269 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:55:11.902597 1269 hybrid_clock.cc:648] HybridClock initialized: now 1786208111902578 us; error 32 us; skew 500 ppm
Aug 08 16:55:11 dist-test-slave-6zbq krb5kdc[316](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208111, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:55:11.905428 1269 init.cc:377] Logged in from keytab as kudu/127.8.24.1@KRBTEST.COM (short username kudu)
I20260808 16:55:11.905934 1269 webserver.cc:460] Webserver started at http://127.8.24.1:33579/ using document root <none> and password file <none>
I20260808 16:55:11.906199 1269 fs_manager.cc:359] Metadata directory not provided
I20260808 16:55:11.906277 1269 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:55:11.907898 1269 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 16:55:11.908610 1319 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:55:11.908763 1269 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 16:55:11.908834 1269 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-0/wal
uuid: "1d47cc8d97044f9aa2943ce89c4efaaa"
format_stamp: "Formatted at 2026-08-08 16:54:51 on dist-test-slave-6zbq"
server_key: "f2d1cdd77f20479ac7b19045512bb25d"
server_key_iv: "703017eb4f461fbc6dfd65dac530115c"
server_key_version: "encryptionkey@0"
I20260808 16:55:11.909099 1269 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:55:11.919528 1269 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:55:11.921550 1269 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:55:11.921679 1269 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:55:11.921856 1269 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:55:11.922221 1328 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 16:55:11.923931 1269 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 16:55:11.923971 1269 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260808 16:55:11.924000 1269 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 16:55:11.924539 1269 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 16:55:11.924571 1269 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260808 16:55:11.924659 1328 tablet_bootstrap.cc:492] T 2cd8f81ed87443b6b656616784da4846 P 1d47cc8d97044f9aa2943ce89c4efaaa: Bootstrap starting.
I20260808 16:55:11.930956 1269 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.1:37163
I20260808 16:55:11.931274 1269 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-0/data/info.pb
I20260808 16:55:11.935102 1505 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.1:37163 every 8 connection(s)
I20260808 16:55:11.935402 1328 log.cc:824] T 2cd8f81ed87443b6b656616784da4846 P 1d47cc8d97044f9aa2943ce89c4efaaa: Log is configured to *not* fsync() on all Append() calls
I20260808 16:55:11.937140 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 1269
I20260808 16:55:11.937407 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.2:44295
--local_ip_for_outbound_sockets=127.8.24.2
--tserver_master_addrs=127.8.24.62:36933
--webserver_port=45671
--webserver_interface=127.8.24.2
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.2.keytab
--principal=kudu/127.8.24.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:36953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 16:55:11.937798 1328 tablet_bootstrap.cc:492] T 2cd8f81ed87443b6b656616784da4846 P 1d47cc8d97044f9aa2943ce89c4efaaa: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 16:55:11.938055 1328 tablet_bootstrap.cc:492] T 2cd8f81ed87443b6b656616784da4846 P 1d47cc8d97044f9aa2943ce89c4efaaa: Bootstrap complete.
I20260808 16:55:11.939142 1328 ts_tablet_manager.cc:1397] T 2cd8f81ed87443b6b656616784da4846 P 1d47cc8d97044f9aa2943ce89c4efaaa: Time spent bootstrapping tablet: real 0.015s user 0.008s sys 0.006s
I20260808 16:55:11.940385 1328 raft_consensus.cc:348] T 2cd8f81ed87443b6b656616784da4846 P 1d47cc8d97044f9aa2943ce89c4efaaa [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "275153aa50f44d19b906434156c28e31" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 44295 } } peers { permanent_uuid: "d99276ea3dad479c9a077324c1e7dccc" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 33733 } } peers { permanent_uuid: "1d47cc8d97044f9aa2943ce89c4efaaa" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 37163 } }
I20260808 16:55:11.940763 1328 raft_consensus.cc:729] T 2cd8f81ed87443b6b656616784da4846 P 1d47cc8d97044f9aa2943ce89c4efaaa [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d47cc8d97044f9aa2943ce89c4efaaa, State: Initialized, Role: FOLLOWER
I20260808 16:55:11.940949 1328 consensus_queue.cc:260] T 2cd8f81ed87443b6b656616784da4846 P 1d47cc8d97044f9aa2943ce89c4efaaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "275153aa50f44d19b906434156c28e31" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 44295 } } peers { permanent_uuid: "d99276ea3dad479c9a077324c1e7dccc" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 33733 } } peers { permanent_uuid: "1d47cc8d97044f9aa2943ce89c4efaaa" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 37163 } }
I20260808 16:55:11.941228 1328 ts_tablet_manager.cc:1428] T 2cd8f81ed87443b6b656616784da4846 P 1d47cc8d97044f9aa2943ce89c4efaaa: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
Aug 08 16:55:11 dist-test-slave-6zbq krb5kdc[316](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208111, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.1@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260808 16:55:11.944670 1508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:11.931858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:36933 (local address 127.8.24.1:59961)
0808 16:55:11.934525 (+ 2667us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:11.934560 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 16:55:11.934944 (+ 384us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:55:11.935767 (+ 823us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:55:11.935771 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:55:11.936033 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:55:11.936210 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:11.936215 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:11.937487 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:11.937488 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:11.938620 (+ 1132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:11.938623 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:11.939524 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:11.939526 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:11.939676 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:11.940149 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:55:11.940159 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:55:11.941615 (+ 1456us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:55:11.943248 (+ 1633us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:55:11.943255 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:55:11.943257 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:55:11.943457 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:55:11.943677 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:55:11.943678 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:55:11.943678 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:55:11.943779 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:55:11.944103 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:55:11.944109 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:55:11.944235 (+ 126us) client_negotiation.cc:769] Sending connection context
0808 16:55:11.944317 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 16:55:11.944475 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2628,"thread_start_us":96,"threads_started":1}
I20260808 16:55:11.944670 1509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:11.931924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:59961 (local address 127.8.24.62:36933)
0808 16:55:11.935311 (+ 3387us) server_negotiation.cc:206] Beginning negotiation
0808 16:55:11.935319 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 16:55:11.935352 (+ 33us) server_negotiation.cc:407] Connection header received
0808 16:55:11.935450 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:55:11.935455 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:55:11.935505 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:55:11.935632 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:55:11.936322 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:11.937345 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:11.938740 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:11.939313 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:11.939413 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:11.941717 (+ 2304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:55:11.941721 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:55:11.941722 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:55:11.941732 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:55:11.943185 (+ 1453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:55:11.943528 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:55:11.943531 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:55:11.943532 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:55:11.943617 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:55:11.943834 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:55:11.943835 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:55:11.943836 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:55:11.944041 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:55:11.944114 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 16:55:11.944380 (+ 266us) server_negotiation.cc:299] Negotiation successful
0808 16:55:11.944498 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3339,"thread_start_us":74,"threads_started":1}
I20260808 16:55:11.945770 1506 heartbeater.cc:347] Connected to a master server at 127.8.24.62:36933
I20260808 16:55:11.945861 1506 heartbeater.cc:464] Registering TS with master...
I20260808 16:55:11.946022 1506 heartbeater.cc:511] Master 127.8.24.62:36933 requested a full tablet report, sending...
I20260808 16:55:11.946679 1205 ts_manager.cc:194] Registered new tserver with Master: 1d47cc8d97044f9aa2943ce89c4efaaa (127.8.24.1:37163)
I20260808 16:55:11.947666 1205 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.1@KRBTEST.COM'} at 127.8.24.1:59961
I20260808 16:55:11.948520 1506 heartbeater.cc:503] Master 127.8.24.62:36933 was elected leader, sending a full tablet report...
I20260808 16:55:11.953202 1520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:11.947434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46248 (local address 127.8.24.1:37163)
0808 16:55:11.947570 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 16:55:11.947574 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 16:55:11.950522 (+ 2948us) server_negotiation.cc:407] Connection header received
0808 16:55:11.950595 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:55:11.950598 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:55:11.950611 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:55:11.950685 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:55:11.950814 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:11.951490 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:11.952406 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:11.952844 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:11.952902 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:11.952951 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 16:55:11.953105 (+ 154us) server_negotiation.cc:299] Negotiation successful
0808 16:55:11.953140 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":75,"threads_started":1}
I20260808 16:55:11.953208 1519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:11.947206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:37163 (local address 127.0.0.1:46248)
0808 16:55:11.950434 (+ 3228us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:11.950461 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0808 16:55:11.950526 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:55:11.950694 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:55:11.950697 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:55:11.950698 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:55:11.950753 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:11.950755 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:11.951560 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:11.951561 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:11.952330 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:11.952332 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:11.952915 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:11.952916 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:11.953014 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:11.953027 (+ 13us) client_negotiation.cc:769] Sending connection context
0808 16:55:11.953084 (+ 57us) client_negotiation.cc:240] Negotiation successful
0808 16:55:11.953144 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3200,"thread_start_us":79,"threads_started":1}
I20260808 16:55:11.953672 1382 tablet_service.cc:1514] Processing DeleteTablet for tablet 2cd8f81ed87443b6b656616784da4846 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:54:51 UTC) from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:46248
I20260808 16:55:11.954077 1522 tablet_replica.cc:330] stopping tablet replica
I20260808 16:55:11.954185 1522 raft_consensus.cc:2229] T 2cd8f81ed87443b6b656616784da4846 P 1d47cc8d97044f9aa2943ce89c4efaaa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 16:55:11.954247 1522 raft_consensus.cc:2258] T 2cd8f81ed87443b6b656616784da4846 P 1d47cc8d97044f9aa2943ce89c4efaaa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:55:11.954519 1522 ts_tablet_manager.cc:1905] T 2cd8f81ed87443b6b656616784da4846 P 1d47cc8d97044f9aa2943ce89c4efaaa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:55:11.956326 1522 ts_tablet_manager.cc:1918] T 2cd8f81ed87443b6b656616784da4846 P 1d47cc8d97044f9aa2943ce89c4efaaa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 16:55:11.956387 1522 log.cc:1196] T 2cd8f81ed87443b6b656616784da4846 P 1d47cc8d97044f9aa2943ce89c4efaaa: Deleting WAL directory at /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-0/wal/wals/2cd8f81ed87443b6b656616784da4846
I20260808 16:55:11.956580 1522 ts_tablet_manager.cc:1939] T 2cd8f81ed87443b6b656616784da4846 P 1d47cc8d97044f9aa2943ce89c4efaaa: Deleting consensus metadata
I20260808 16:55:11.957000 1174 catalog_manager.cc:4841] TS 1d47cc8d97044f9aa2943ce89c4efaaa (127.8.24.1:37163): tablet 2cd8f81ed87443b6b656616784da4846 (table test-table [id=e0a2f2f3fa184a028eebb7219bb5f13e]) successfully deleted
W20260808 16:55:12.021801 1514 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:55:12.021974 1514 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:55:12.021996 1514 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:55:12.023135 1514 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:55:12.023176 1514 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:55:12.023201 1514 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.2
I20260808 16:55:12.024595 1514 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:36953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.2.keytab
--principal=kudu/127.8.24.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.2:44295
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.24.2
--webserver_port=45671
--tserver_master_addrs=127.8.24.62:36933
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1514
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.2
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:55:12.024793 1514 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:55:12.024987 1514 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:55:12.029573 1534 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:55:12.029539 1530 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:55:12.029551 1531 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:55:12.029584 1514 server_base.cc:1034] running on GCE node
I20260808 16:55:12.029948 1514 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:55:12.030157 1514 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:55:12.031283 1514 hybrid_clock.cc:648] HybridClock initialized: now 1786208112031270 us; error 29 us; skew 500 ppm
Aug 08 16:55:12 dist-test-slave-6zbq krb5kdc[316](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208112, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:55:12.033465 1514 init.cc:377] Logged in from keytab as kudu/127.8.24.2@KRBTEST.COM (short username kudu)
I20260808 16:55:12.033844 1514 webserver.cc:460] Webserver started at http://127.8.24.2:45671/ using document root <none> and password file <none>
I20260808 16:55:12.034047 1514 fs_manager.cc:359] Metadata directory not provided
I20260808 16:55:12.034104 1514 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:55:12.035372 1514 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 16:55:12.035899 1551 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:55:12.036036 1514 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 16:55:12.036113 1514 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-1/wal
uuid: "275153aa50f44d19b906434156c28e31"
format_stamp: "Formatted at 2026-08-08 16:54:51 on dist-test-slave-6zbq"
server_key: "daa72f3463c2e38c9867de56b3e7cd98"
server_key_iv: "c7125c51b2b17114d39058e2334f00d9"
server_key_version: "encryptionkey@0"
I20260808 16:55:12.036414 1514 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:55:12.050942 1514 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:55:12.052487 1514 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:55:12.052605 1514 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:55:12.052793 1514 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:55:12.053049 1514 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:55:12.053081 1514 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:55:12.053108 1514 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:55:12.053133 1514 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:55:12.058730 1514 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.2:44295
I20260808 16:55:12.058773 1724 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.2:44295 every 8 connection(s)
I20260808 16:55:12.059068 1514 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-1/data/info.pb
Aug 08 16:55:12 dist-test-slave-6zbq krb5kdc[316](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208112, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.2@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260808 16:55:12.064586 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 1514
I20260808 16:55:12.064842 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.3:33733
--local_ip_for_outbound_sockets=127.8.24.3
--tserver_master_addrs=127.8.24.62:36933
--webserver_port=39907
--webserver_interface=127.8.24.3
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.3.keytab
--principal=kudu/127.8.24.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:36953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 16:55:12.067651 1509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:12.059675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:44545 (local address 127.8.24.62:36933)
0808 16:55:12.059729 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 16:55:12.059731 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:55:12.059912 (+ 181us) server_negotiation.cc:407] Connection header received
0808 16:55:12.060363 (+ 451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:55:12.060364 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:55:12.060377 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:55:12.060423 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:55:12.061308 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:12.061794 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:12.062747 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:12.062962 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:12.063017 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:12.065055 (+ 2038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:55:12.065056 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:55:12.065057 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:55:12.065065 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:55:12.066409 (+ 1344us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:55:12.066852 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:55:12.066853 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:55:12.066853 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:55:12.066902 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:55:12.067126 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:55:12.067126 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:55:12.067127 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:55:12.067214 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:55:12.067265 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 16:55:12.067529 (+ 264us) server_negotiation.cc:299] Negotiation successful
0808 16:55:12.067590 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 16:55:12.067885 1728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:12.059653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:36933 (local address 127.8.24.2:44545)
0808 16:55:12.059806 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:12.059839 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 16:55:12.060274 (+ 435us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:55:12.060498 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:55:12.060501 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:55:12.061024 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:55:12.061213 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:12.061218 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:12.061875 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:12.061876 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:12.062670 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:12.062672 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:12.063041 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:12.063042 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:12.063146 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:12.063521 (+ 375us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:55:12.063529 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:55:12.064929 (+ 1400us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:55:12.066500 (+ 1571us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:55:12.066508 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:55:12.066511 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:55:12.066779 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:55:12.066955 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:55:12.066955 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:55:12.066955 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:55:12.067075 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:55:12.067268 (+ 193us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:55:12.067273 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:55:12.067415 (+ 142us) client_negotiation.cc:769] Sending connection context
0808 16:55:12.067519 (+ 104us) client_negotiation.cc:240] Negotiation successful
0808 16:55:12.067676 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":78,"threads_started":1}
I20260808 16:55:12.068313 1725 heartbeater.cc:347] Connected to a master server at 127.8.24.62:36933
I20260808 16:55:12.068392 1725 heartbeater.cc:464] Registering TS with master...
I20260808 16:55:12.068554 1725 heartbeater.cc:511] Master 127.8.24.62:36933 requested a full tablet report, sending...
I20260808 16:55:12.068892 1205 ts_manager.cc:194] Registered new tserver with Master: 275153aa50f44d19b906434156c28e31 (127.8.24.2:44295)
I20260808 16:55:12.069238 1205 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.2@KRBTEST.COM'} at 127.8.24.2:44545
W20260808 16:55:12.148901 1729 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:55:12.149075 1729 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:55:12.149096 1729 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:55:12.150151 1729 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:55:12.150197 1729 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:55:12.150240 1729 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.3
I20260808 16:55:12.151664 1729 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:36953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.3.keytab
--principal=kudu/127.8.24.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.3:33733
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.24.3
--webserver_port=39907
--tserver_master_addrs=127.8.24.62:36933
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1729
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.3
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:55:12.151841 1729 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:55:12.152001 1729 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:55:12.156416 1738 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:55:12.156451 1740 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:55:12.156454 1729 server_base.cc:1034] running on GCE node
W20260808 16:55:12.156423 1737 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:55:12.156772 1729 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:55:12.156963 1729 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:55:12.158105 1729 hybrid_clock.cc:648] HybridClock initialized: now 1786208112158092 us; error 31 us; skew 500 ppm
Aug 08 16:55:12 dist-test-slave-6zbq krb5kdc[316](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208112, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:55:12.160322 1729 init.cc:377] Logged in from keytab as kudu/127.8.24.3@KRBTEST.COM (short username kudu)
I20260808 16:55:12.160713 1729 webserver.cc:460] Webserver started at http://127.8.24.3:39907/ using document root <none> and password file <none>
I20260808 16:55:12.160920 1729 fs_manager.cc:359] Metadata directory not provided
I20260808 16:55:12.160979 1729 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:55:12.162169 1729 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 16:55:12.162708 1753 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:55:12.162847 1729 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 16:55:12.162925 1729 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-2/wal
uuid: "d99276ea3dad479c9a077324c1e7dccc"
format_stamp: "Formatted at 2026-08-08 16:54:51 on dist-test-slave-6zbq"
server_key: "4f814eb022fcb67fa8f59f0e700aca0b"
server_key_iv: "4f681b16dacc81905593ce21969b24df"
server_key_version: "encryptionkey@0"
I20260808 16:55:12.163195 1729 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:55:12.179060 1729 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:55:12.180686 1729 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:55:12.180819 1729 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:55:12.181025 1729 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:55:12.181298 1729 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:55:12.181330 1729 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:55:12.181356 1729 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:55:12.181373 1729 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:55:12.186633 1729 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.3:33733
I20260808 16:55:12.186674 1922 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.3:33733 every 8 connection(s)
I20260808 16:55:12.186975 1729 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-2/data/info.pb
I20260808 16:55:12.190750 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 1729
Aug 08 16:55:12 dist-test-slave-6zbq krb5kdc[316](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208112, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.3@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260808 16:55:12.195060 1509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:12.187555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:53865 (local address 127.8.24.62:36933)
0808 16:55:12.187635 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0808 16:55:12.187638 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 16:55:12.187794 (+ 156us) server_negotiation.cc:407] Connection header received
0808 16:55:12.188185 (+ 391us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:55:12.188186 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:55:12.188196 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:55:12.188241 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:55:12.189043 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:12.189519 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:12.190416 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:12.190611 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:12.190669 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:12.192576 (+ 1907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:55:12.192577 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:55:12.192578 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:55:12.192585 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:55:12.193849 (+ 1264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:55:12.194202 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:55:12.194203 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:55:12.194203 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:55:12.194256 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:55:12.194504 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:55:12.194505 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:55:12.194505 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:55:12.194611 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:55:12.194665 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 16:55:12.194941 (+ 276us) server_negotiation.cc:299] Negotiation successful
0808 16:55:12.194999 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 16:55:12.195256 1927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:12.187535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:36933 (local address 127.8.24.3:53865)
0808 16:55:12.187678 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:12.187710 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0808 16:55:12.188113 (+ 403us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:55:12.188293 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:55:12.188296 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:55:12.188804 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:55:12.188973 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:12.188978 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:12.189576 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:12.189577 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:12.190309 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:12.190311 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:12.190679 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:12.190680 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:12.190781 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:12.191238 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:55:12.191248 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:55:12.192490 (+ 1242us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:55:12.193928 (+ 1438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:55:12.193934 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:55:12.193936 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:55:12.194138 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:55:12.194310 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:55:12.194310 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:55:12.194311 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:55:12.194448 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:55:12.194663 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:55:12.194669 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:55:12.194814 (+ 145us) client_negotiation.cc:769] Sending connection context
0808 16:55:12.194924 (+ 110us) client_negotiation.cc:240] Negotiation successful
0808 16:55:12.195050 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":76,"threads_started":1}
I20260808 16:55:12.195688 1925 heartbeater.cc:347] Connected to a master server at 127.8.24.62:36933
I20260808 16:55:12.195782 1925 heartbeater.cc:464] Registering TS with master...
I20260808 16:55:12.195956 1925 heartbeater.cc:511] Master 127.8.24.62:36933 requested a full tablet report, sending...
I20260808 16:55:12.196321 1205 ts_manager.cc:194] Registered new tserver with Master: d99276ea3dad479c9a077324c1e7dccc (127.8.24.3:33733)
I20260808 16:55:12.196790 1205 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.3@KRBTEST.COM'} at 127.8.24.3:53865
I20260808 16:55:12.197175 1929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:12.191195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48544 (local address 127.8.24.62:36933)
0808 16:55:12.191327 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 16:55:12.191329 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:55:12.191336 (+ 7us) server_negotiation.cc:407] Connection header received
0808 16:55:12.191356 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:55:12.191356 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:55:12.191366 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:55:12.191418 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:55:12.191978 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:12.192480 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:12.193215 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:12.193402 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:12.193452 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:12.194518 (+ 1066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:55:12.194520 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:55:12.194520 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:55:12.194527 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:55:12.196138 (+ 1611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:55:12.196494 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:55:12.196495 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:55:12.196495 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:55:12.196546 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:55:12.196707 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:55:12.196708 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:55:12.196708 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:55:12.196796 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:55:12.196948 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0808 16:55:12.197011 (+ 63us) server_negotiation.cc:299] Negotiation successful
0808 16:55:12.197091 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":112,"thread_start_us":70,"threads_started":1}
I20260808 16:55:12.197840 8288 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 16:55:12.200119 1929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:12.198153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48558 (local address 127.8.24.62:36933)
0808 16:55:12.198190 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 16:55:12.198192 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:55:12.198245 (+ 53us) server_negotiation.cc:407] Connection header received
0808 16:55:12.198297 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:55:12.198298 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:55:12.198326 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:55:12.198418 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 16:55:12.198497 (+ 79us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:12.198909 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:12.199547 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:12.199711 (+ 164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:12.199747 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:12.199839 (+ 92us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 16:55:12.199871 (+ 32us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0808 16:55:12.200067 (+ 196us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:48558: token expired
Metrics: {"server-negotiator.queue_time_us":17}
W20260808 16:55:12.200176 1929 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:48558: token expired
W20260808 16:55:12.200181 1100 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.8.24.62:36933)
I20260808 16:55:12.205262 1929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:12.200402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48566 (local address 127.8.24.62:36933)
0808 16:55:12.200436 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0808 16:55:12.200437 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 16:55:12.200446 (+ 9us) server_negotiation.cc:407] Connection header received
0808 16:55:12.200466 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:55:12.200466 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:55:12.200474 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:55:12.200518 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:55:12.200927 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:12.201364 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:12.201998 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:12.202175 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:12.202225 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:12.203185 (+ 960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:55:12.203186 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:55:12.203187 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:55:12.203193 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:55:12.204330 (+ 1137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:55:12.204613 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:55:12.204614 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:55:12.204615 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:55:12.204663 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:55:12.204819 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:55:12.204820 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:55:12.204820 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:55:12.204938 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:55:12.205022 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 16:55:12.205120 (+ 98us) server_negotiation.cc:299] Negotiation successful
0808 16:55:12.205187 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 16:55:12.206185 1205 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48566:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 16:55:12.207691 1205 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 16:55:12.209649 1382 tablet_service.cc:1467] Processing CreateTablet for tablet 1040396622f846c5936ecc5b21bada0b (DEFAULT_TABLE table=test-table [id=15d36293befd42b8a42a3700a278ef3d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:55:12.209817 1382 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1040396622f846c5936ecc5b21bada0b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:55:12.211092 1328 tablet_bootstrap.cc:492] T 1040396622f846c5936ecc5b21bada0b P 1d47cc8d97044f9aa2943ce89c4efaaa: Bootstrap starting.
I20260808 16:55:12.211433 1328 tablet_bootstrap.cc:654] T 1040396622f846c5936ecc5b21bada0b P 1d47cc8d97044f9aa2943ce89c4efaaa: Neither blocks nor log segments found. Creating new log.
I20260808 16:55:12.211971 1328 tablet_bootstrap.cc:492] T 1040396622f846c5936ecc5b21bada0b P 1d47cc8d97044f9aa2943ce89c4efaaa: No bootstrap required, opened a new log
I20260808 16:55:12.212023 1328 ts_tablet_manager.cc:1397] T 1040396622f846c5936ecc5b21bada0b P 1d47cc8d97044f9aa2943ce89c4efaaa: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260808 16:55:12.212118 1328 raft_consensus.cc:348] T 1040396622f846c5936ecc5b21bada0b P 1d47cc8d97044f9aa2943ce89c4efaaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "275153aa50f44d19b906434156c28e31" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 44295 } } peers { permanent_uuid: "d99276ea3dad479c9a077324c1e7dccc" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 33733 } } peers { permanent_uuid: "1d47cc8d97044f9aa2943ce89c4efaaa" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 37163 } }
I20260808 16:55:12.212190 1328 raft_consensus.cc:374] T 1040396622f846c5936ecc5b21bada0b P 1d47cc8d97044f9aa2943ce89c4efaaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:55:12.212245 1328 raft_consensus.cc:729] T 1040396622f846c5936ecc5b21bada0b P 1d47cc8d97044f9aa2943ce89c4efaaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d47cc8d97044f9aa2943ce89c4efaaa, State: Initialized, Role: FOLLOWER
I20260808 16:55:12.212285 1328 consensus_queue.cc:260] T 1040396622f846c5936ecc5b21bada0b P 1d47cc8d97044f9aa2943ce89c4efaaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "275153aa50f44d19b906434156c28e31" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 44295 } } peers { permanent_uuid: "d99276ea3dad479c9a077324c1e7dccc" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 33733 } } peers { permanent_uuid: "1d47cc8d97044f9aa2943ce89c4efaaa" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 37163 } }
I20260808 16:55:12.212383 1328 ts_tablet_manager.cc:1428] T 1040396622f846c5936ecc5b21bada0b P 1d47cc8d97044f9aa2943ce89c4efaaa: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 16:55:12.213140 1519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:12.209250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.2:44295 (local address 127.0.0.1:56506)
0808 16:55:12.209294 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:12.209303 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 16:55:12.209336 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:55:12.209750 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:55:12.209751 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:55:12.209752 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:55:12.209786 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:12.209787 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:12.211035 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:12.211037 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:12.212006 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:12.212010 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:12.212930 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:12.212931 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:12.212988 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:12.212991 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 16:55:12.213043 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 16:55:12.213082 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260808 16:55:12.213238 1934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:12.209427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56506 (local address 127.8.24.2:44295)
0808 16:55:12.209572 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0808 16:55:12.209577 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 16:55:12.209588 (+ 11us) server_negotiation.cc:407] Connection header received
0808 16:55:12.209628 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:55:12.209630 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:55:12.209645 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:55:12.209729 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:55:12.209847 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:12.210950 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:12.212100 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:12.212816 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:12.212889 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:12.212948 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 16:55:12.213115 (+ 167us) server_negotiation.cc:299] Negotiation successful
0808 16:55:12.213157 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":77,"threads_started":1}
I20260808 16:55:12.213284 1931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:12.209281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.3:33733 (local address 127.0.0.1:50788)
0808 16:55:12.209409 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:12.209418 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 16:55:12.209461 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:55:12.209864 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:55:12.209865 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:55:12.209867 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:55:12.209894 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:12.209896 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:12.211072 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:12.211073 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:12.212101 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:12.212104 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:12.213042 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:12.213043 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:12.213127 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:12.213131 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 16:55:12.213174 (+ 43us) client_negotiation.cc:240] Negotiation successful
0808 16:55:12.213209 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"mutex_wait_us":1,"thread_start_us":82,"threads_started":1}
I20260808 16:55:12.213476 1932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:12.209360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50788 (local address 127.8.24.3:33733)
0808 16:55:12.209602 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0808 16:55:12.209606 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 16:55:12.209614 (+ 8us) server_negotiation.cc:407] Connection header received
0808 16:55:12.209642 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:55:12.209643 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:55:12.209655 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:55:12.209729 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:55:12.209954 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:12.210987 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:12.212172 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:12.212912 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:12.212965 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:12.213007 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0808 16:55:12.213373 (+ 366us) server_negotiation.cc:299] Negotiation successful
0808 16:55:12.213411 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":217,"thread_start_us":72,"threads_started":1}
I20260808 16:55:12.213933 1624 tablet_service.cc:1467] Processing CreateTablet for tablet 1040396622f846c5936ecc5b21bada0b (DEFAULT_TABLE table=test-table [id=15d36293befd42b8a42a3700a278ef3d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:55:12.213960 1825 tablet_service.cc:1467] Processing CreateTablet for tablet 1040396622f846c5936ecc5b21bada0b (DEFAULT_TABLE table=test-table [id=15d36293befd42b8a42a3700a278ef3d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:55:12.214197 1825 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1040396622f846c5936ecc5b21bada0b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:55:12.214207 1624 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1040396622f846c5936ecc5b21bada0b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:55:12.216642 1939 tablet_bootstrap.cc:492] T 1040396622f846c5936ecc5b21bada0b P d99276ea3dad479c9a077324c1e7dccc: Bootstrap starting.
I20260808 16:55:12.216605 1937 tablet_bootstrap.cc:492] T 1040396622f846c5936ecc5b21bada0b P 275153aa50f44d19b906434156c28e31: Bootstrap starting.
I20260808 16:55:12.217212 1939 tablet_bootstrap.cc:654] T 1040396622f846c5936ecc5b21bada0b P d99276ea3dad479c9a077324c1e7dccc: Neither blocks nor log segments found. Creating new log.
I20260808 16:55:12.217242 1937 tablet_bootstrap.cc:654] T 1040396622f846c5936ecc5b21bada0b P 275153aa50f44d19b906434156c28e31: Neither blocks nor log segments found. Creating new log.
I20260808 16:55:12.217641 1939 log.cc:824] T 1040396622f846c5936ecc5b21bada0b P d99276ea3dad479c9a077324c1e7dccc: Log is configured to *not* fsync() on all Append() calls
I20260808 16:55:12.217682 1937 log.cc:824] T 1040396622f846c5936ecc5b21bada0b P 275153aa50f44d19b906434156c28e31: Log is configured to *not* fsync() on all Append() calls
I20260808 16:55:12.218295 1939 tablet_bootstrap.cc:492] T 1040396622f846c5936ecc5b21bada0b P d99276ea3dad479c9a077324c1e7dccc: No bootstrap required, opened a new log
I20260808 16:55:12.218297 1937 tablet_bootstrap.cc:492] T 1040396622f846c5936ecc5b21bada0b P 275153aa50f44d19b906434156c28e31: No bootstrap required, opened a new log
I20260808 16:55:12.218375 1937 ts_tablet_manager.cc:1397] T 1040396622f846c5936ecc5b21bada0b P 275153aa50f44d19b906434156c28e31: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 16:55:12.218381 1939 ts_tablet_manager.cc:1397] T 1040396622f846c5936ecc5b21bada0b P d99276ea3dad479c9a077324c1e7dccc: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 16:55:12.219758 1939 raft_consensus.cc:348] T 1040396622f846c5936ecc5b21bada0b P d99276ea3dad479c9a077324c1e7dccc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "275153aa50f44d19b906434156c28e31" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 44295 } } peers { permanent_uuid: "d99276ea3dad479c9a077324c1e7dccc" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 33733 } } peers { permanent_uuid: "1d47cc8d97044f9aa2943ce89c4efaaa" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 37163 } }
I20260808 16:55:12.219899 1939 raft_consensus.cc:374] T 1040396622f846c5936ecc5b21bada0b P d99276ea3dad479c9a077324c1e7dccc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:55:12.219944 1939 raft_consensus.cc:729] T 1040396622f846c5936ecc5b21bada0b P d99276ea3dad479c9a077324c1e7dccc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d99276ea3dad479c9a077324c1e7dccc, State: Initialized, Role: FOLLOWER
I20260808 16:55:12.219884 1937 raft_consensus.cc:348] T 1040396622f846c5936ecc5b21bada0b P 275153aa50f44d19b906434156c28e31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "275153aa50f44d19b906434156c28e31" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 44295 } } peers { permanent_uuid: "d99276ea3dad479c9a077324c1e7dccc" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 33733 } } peers { permanent_uuid: "1d47cc8d97044f9aa2943ce89c4efaaa" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 37163 } }
I20260808 16:55:12.220033 1937 raft_consensus.cc:374] T 1040396622f846c5936ecc5b21bada0b P 275153aa50f44d19b906434156c28e31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:55:12.220067 1937 raft_consensus.cc:729] T 1040396622f846c5936ecc5b21bada0b P 275153aa50f44d19b906434156c28e31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 275153aa50f44d19b906434156c28e31, State: Initialized, Role: FOLLOWER
I20260808 16:55:12.220062 1939 consensus_queue.cc:260] T 1040396622f846c5936ecc5b21bada0b P d99276ea3dad479c9a077324c1e7dccc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "275153aa50f44d19b906434156c28e31" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 44295 } } peers { permanent_uuid: "d99276ea3dad479c9a077324c1e7dccc" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 33733 } } peers { permanent_uuid: "1d47cc8d97044f9aa2943ce89c4efaaa" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 37163 } }
I20260808 16:55:12.220173 1937 consensus_queue.cc:260] T 1040396622f846c5936ecc5b21bada0b P 275153aa50f44d19b906434156c28e31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "275153aa50f44d19b906434156c28e31" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 44295 } } peers { permanent_uuid: "d99276ea3dad479c9a077324c1e7dccc" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 33733 } } peers { permanent_uuid: "1d47cc8d97044f9aa2943ce89c4efaaa" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 37163 } }
I20260808 16:55:12.220328 1939 ts_tablet_manager.cc:1428] T 1040396622f846c5936ecc5b21bada0b P d99276ea3dad479c9a077324c1e7dccc: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 16:55:12.220423 1925 heartbeater.cc:503] Master 127.8.24.62:36933 was elected leader, sending a full tablet report...
I20260808 16:55:12.220443 1725 heartbeater.cc:503] Master 127.8.24.62:36933 was elected leader, sending a full tablet report...
I20260808 16:55:12.220451 1937 ts_tablet_manager.cc:1428] T 1040396622f846c5936ecc5b21bada0b P 275153aa50f44d19b906434156c28e31: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 16:55:12.227294 1943 raft_consensus.cc:482] T 1040396622f846c5936ecc5b21bada0b P d99276ea3dad479c9a077324c1e7dccc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 16:55:12.227377 1943 raft_consensus.cc:504] T 1040396622f846c5936ecc5b21bada0b P d99276ea3dad479c9a077324c1e7dccc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "275153aa50f44d19b906434156c28e31" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 44295 } } peers { permanent_uuid: "d99276ea3dad479c9a077324c1e7dccc" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 33733 } } peers { permanent_uuid: "1d47cc8d97044f9aa2943ce89c4efaaa" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 37163 } }
I20260808 16:55:12.227638 1943 leader_election.cc:288] T 1040396622f846c5936ecc5b21bada0b P d99276ea3dad479c9a077324c1e7dccc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 275153aa50f44d19b906434156c28e31 (127.8.24.2:44295), 1d47cc8d97044f9aa2943ce89c4efaaa (127.8.24.1:37163)
I20260808 16:55:12.230160 1934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:12.227823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:38943 (local address 127.8.24.2:44295)
0808 16:55:12.227914 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0808 16:55:12.227915 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 16:55:12.227922 (+ 7us) server_negotiation.cc:407] Connection header received
0808 16:55:12.227942 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:55:12.227943 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:55:12.227951 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:55:12.228000 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:55:12.228114 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:12.228535 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:12.229503 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:12.229882 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:12.229942 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:12.229951 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 16:55:12.230076 (+ 125us) server_negotiation.cc:299] Negotiation successful
0808 16:55:12.230106 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260808 16:55:12.230163 1927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:12.227756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.2:44295 (local address 127.8.24.3:38943)
0808 16:55:12.227808 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:12.227818 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 16:55:12.227851 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:55:12.228013 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:55:12.228014 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:55:12.228016 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:55:12.228064 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:12.228065 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:12.228599 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:12.228600 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:12.229434 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:12.229436 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:12.229950 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:12.229950 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:12.230020 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:12.230023 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 16:55:12.230066 (+ 43us) client_negotiation.cc:240] Negotiation successful
0808 16:55:12.230101 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260808 16:55:12.230221 1520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:12.227824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:47405 (local address 127.8.24.1:37163)
0808 16:55:12.227865 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 16:55:12.227867 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:55:12.227949 (+ 82us) server_negotiation.cc:407] Connection header received
0808 16:55:12.227996 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:55:12.227997 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:55:12.228007 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:55:12.228056 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:55:12.228174 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:12.228603 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:12.229590 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:12.229941 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:12.229998 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:12.230006 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 16:55:12.230128 (+ 122us) server_negotiation.cc:299] Negotiation successful
0808 16:55:12.230164 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 16:55:12.230265 1950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:12.227761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:37163 (local address 127.8.24.3:47405)
0808 16:55:12.227899 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:12.227908 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 16:55:12.227953 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:55:12.228060 (+ 107us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:55:12.228061 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:55:12.228062 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:55:12.228119 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:12.228121 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:12.228667 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:12.228668 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:12.229521 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:12.229524 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:12.230007 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:12.230008 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:12.230072 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:12.230075 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 16:55:12.230118 (+ 43us) client_negotiation.cc:240] Negotiation successful
0808 16:55:12.230153 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"mutex_wait_us":19,"thread_start_us":78,"threads_started":1}
I20260808 16:55:12.230613 1655 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1040396622f846c5936ecc5b21bada0b" candidate_uuid: "d99276ea3dad479c9a077324c1e7dccc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "275153aa50f44d19b906434156c28e31" is_pre_election: true
I20260808 16:55:12.230731 1655 raft_consensus.cc:2454] T 1040396622f846c5936ecc5b21bada0b P 275153aa50f44d19b906434156c28e31 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d99276ea3dad479c9a077324c1e7dccc in term 0.
I20260808 16:55:12.230721 1439 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1040396622f846c5936ecc5b21bada0b" candidate_uuid: "d99276ea3dad479c9a077324c1e7dccc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d47cc8d97044f9aa2943ce89c4efaaa" is_pre_election: true
I20260808 16:55:12.230873 1439 raft_consensus.cc:2454] T 1040396622f846c5936ecc5b21bada0b P 1d47cc8d97044f9aa2943ce89c4efaaa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d99276ea3dad479c9a077324c1e7dccc in term 0.
I20260808 16:55:12.230935 1757 leader_election.cc:302] T 1040396622f846c5936ecc5b21bada0b P d99276ea3dad479c9a077324c1e7dccc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 275153aa50f44d19b906434156c28e31, d99276ea3dad479c9a077324c1e7dccc; no voters:
I20260808 16:55:12.231097 1943 raft_consensus.cc:2781] T 1040396622f846c5936ecc5b21bada0b P d99276ea3dad479c9a077324c1e7dccc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 16:55:12.231155 1943 raft_consensus.cc:482] T 1040396622f846c5936ecc5b21bada0b P d99276ea3dad479c9a077324c1e7dccc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 16:55:12.231170 1943 raft_consensus.cc:3037] T 1040396622f846c5936ecc5b21bada0b P d99276ea3dad479c9a077324c1e7dccc [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:55:12.231722 1943 raft_consensus.cc:504] T 1040396622f846c5936ecc5b21bada0b P d99276ea3dad479c9a077324c1e7dccc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "275153aa50f44d19b906434156c28e31" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 44295 } } peers { permanent_uuid: "d99276ea3dad479c9a077324c1e7dccc" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 33733 } } peers { permanent_uuid: "1d47cc8d97044f9aa2943ce89c4efaaa" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 37163 } }
I20260808 16:55:12.231828 1943 leader_election.cc:288] T 1040396622f846c5936ecc5b21bada0b P d99276ea3dad479c9a077324c1e7dccc [CANDIDATE]: Term 1 election: Requested vote from peers 275153aa50f44d19b906434156c28e31 (127.8.24.2:44295), 1d47cc8d97044f9aa2943ce89c4efaaa (127.8.24.1:37163)
I20260808 16:55:12.231964 1439 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1040396622f846c5936ecc5b21bada0b" candidate_uuid: "d99276ea3dad479c9a077324c1e7dccc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d47cc8d97044f9aa2943ce89c4efaaa"
I20260808 16:55:12.231997 1655 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1040396622f846c5936ecc5b21bada0b" candidate_uuid: "d99276ea3dad479c9a077324c1e7dccc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "275153aa50f44d19b906434156c28e31"
I20260808 16:55:12.232041 1439 raft_consensus.cc:3037] T 1040396622f846c5936ecc5b21bada0b P 1d47cc8d97044f9aa2943ce89c4efaaa [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:55:12.232054 1655 raft_consensus.cc:3037] T 1040396622f846c5936ecc5b21bada0b P 275153aa50f44d19b906434156c28e31 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:55:12.232657 1655 raft_consensus.cc:2454] T 1040396622f846c5936ecc5b21bada0b P 275153aa50f44d19b906434156c28e31 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d99276ea3dad479c9a077324c1e7dccc in term 1.
I20260808 16:55:12.232805 1757 leader_election.cc:302] T 1040396622f846c5936ecc5b21bada0b P d99276ea3dad479c9a077324c1e7dccc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 275153aa50f44d19b906434156c28e31, d99276ea3dad479c9a077324c1e7dccc; no voters:
I20260808 16:55:12.232898 1943 raft_consensus.cc:2781] T 1040396622f846c5936ecc5b21bada0b P d99276ea3dad479c9a077324c1e7dccc [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:55:12.233022 1439 raft_consensus.cc:2454] T 1040396622f846c5936ecc5b21bada0b P 1d47cc8d97044f9aa2943ce89c4efaaa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d99276ea3dad479c9a077324c1e7dccc in term 1.
I20260808 16:55:12.233013 1943 raft_consensus.cc:686] T 1040396622f846c5936ecc5b21bada0b P d99276ea3dad479c9a077324c1e7dccc [term 1 LEADER]: Becoming Leader. State: Replica: d99276ea3dad479c9a077324c1e7dccc, State: Running, Role: LEADER
I20260808 16:55:12.233109 1943 consensus_queue.cc:237] T 1040396622f846c5936ecc5b21bada0b P d99276ea3dad479c9a077324c1e7dccc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "275153aa50f44d19b906434156c28e31" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 44295 } } peers { permanent_uuid: "d99276ea3dad479c9a077324c1e7dccc" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 33733 } } peers { permanent_uuid: "1d47cc8d97044f9aa2943ce89c4efaaa" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 37163 } }
I20260808 16:55:12.233794 1203 catalog_manager.cc:5495] T 1040396622f846c5936ecc5b21bada0b P d99276ea3dad479c9a077324c1e7dccc reported cstate change: term changed from 0 to 1, leader changed from <none> to d99276ea3dad479c9a077324c1e7dccc (127.8.24.3). New cstate: current_term: 1 leader_uuid: "d99276ea3dad479c9a077324c1e7dccc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "275153aa50f44d19b906434156c28e31" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 44295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d99276ea3dad479c9a077324c1e7dccc" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 33733 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1d47cc8d97044f9aa2943ce89c4efaaa" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 37163 } health_report { overall_health: UNKNOWN } } }
I20260808 16:55:12.243919 1932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:12.242016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50798 (local address 127.8.24.3:33733)
0808 16:55:12.242053 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 16:55:12.242055 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:55:12.242063 (+ 8us) server_negotiation.cc:407] Connection header received
0808 16:55:12.242109 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:55:12.242109 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:55:12.242124 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:55:12.242180 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 16:55:12.242315 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:12.242772 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:12.243400 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:12.243563 (+ 163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:12.243601 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:12.243686 (+ 85us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 16:55:12.243769 (+ 83us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 16:55:12.243818 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 16:55:12.243845 (+ 27us) server_negotiation.cc:299] Negotiation successful
0808 16:55:12.243860 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 16:55:12.245354 1655 raft_consensus.cc:1261] T 1040396622f846c5936ecc5b21bada0b P 275153aa50f44d19b906434156c28e31 [term 1 FOLLOWER]: Refusing update from remote peer d99276ea3dad479c9a077324c1e7dccc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:55:12.245616 1439 raft_consensus.cc:1261] T 1040396622f846c5936ecc5b21bada0b P 1d47cc8d97044f9aa2943ce89c4efaaa [term 1 FOLLOWER]: Refusing update from remote peer d99276ea3dad479c9a077324c1e7dccc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:55:12.245747 1952 consensus_queue.cc:1035] T 1040396622f846c5936ecc5b21bada0b P d99276ea3dad479c9a077324c1e7dccc [LEADER]: Connected to new peer: Peer: permanent_uuid: "275153aa50f44d19b906434156c28e31" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:55:12.245859 1943 consensus_queue.cc:1035] T 1040396622f846c5936ecc5b21bada0b P d99276ea3dad479c9a077324c1e7dccc [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d47cc8d97044f9aa2943ce89c4efaaa" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 37163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:55:12.248265 1958 mvcc.cc:205] Tried to move back new op lower bound from 7316308427754938368 to 7316308427707424768. Current Snapshot: MvccSnapshot[applied={T|T < 7316308427754938368}]
I20260808 16:55:12.248417 1957 mvcc.cc:205] Tried to move back new op lower bound from 7316308427754938368 to 7316308427707424768. Current Snapshot: MvccSnapshot[applied={T|T < 7316308427754938368}]
I20260808 16:55:12.248061 1956 mvcc.cc:205] Tried to move back new op lower bound from 7316308427754938368 to 7316308427707424768. Current Snapshot: MvccSnapshot[applied={T|T < 7316308427754938368}]
I20260808 16:55:12.251179 1203 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48566:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 16:55:12.251279 1203 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48566:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 16:55:12.252012 1203 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a623e9fb44f445dcb72293c81486a593: Sending DeleteTablet for 3 replicas of tablet 1040396622f846c5936ecc5b21bada0b
I20260808 16:55:12.252334 1825 tablet_service.cc:1514] Processing DeleteTablet for tablet 1040396622f846c5936ecc5b21bada0b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:55:12 UTC) from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:50788
I20260808 16:55:12.252367 1382 tablet_service.cc:1514] Processing DeleteTablet for tablet 1040396622f846c5936ecc5b21bada0b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:55:12 UTC) from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:46248
I20260808 16:55:12.252471 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 1269
I20260808 16:55:12.252555 1974 tablet_replica.cc:330] T 1040396622f846c5936ecc5b21bada0b P d99276ea3dad479c9a077324c1e7dccc: stopping tablet replica
I20260808 16:55:12.252571 1624 tablet_service.cc:1514] Processing DeleteTablet for tablet 1040396622f846c5936ecc5b21bada0b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:55:12 UTC) from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:56506
I20260808 16:55:12.252646 1974 raft_consensus.cc:2229] T 1040396622f846c5936ecc5b21bada0b P d99276ea3dad479c9a077324c1e7dccc [term 1 LEADER]: Raft consensus shutting down.
I20260808 16:55:12.252759 1974 raft_consensus.cc:2258] T 1040396622f846c5936ecc5b21bada0b P d99276ea3dad479c9a077324c1e7dccc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:55:12.260923 1974 ts_tablet_manager.cc:1905] T 1040396622f846c5936ecc5b21bada0b P d99276ea3dad479c9a077324c1e7dccc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:55:12.262419 1975 tablet_replica.cc:330] T 1040396622f846c5936ecc5b21bada0b P 275153aa50f44d19b906434156c28e31: stopping tablet replica
I20260808 16:55:12.262606 1975 raft_consensus.cc:2229] T 1040396622f846c5936ecc5b21bada0b P 275153aa50f44d19b906434156c28e31 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 16:55:12.262625 1974 ts_tablet_manager.cc:1918] T 1040396622f846c5936ecc5b21bada0b P d99276ea3dad479c9a077324c1e7dccc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 16:55:12.262773 1974 log.cc:1196] T 1040396622f846c5936ecc5b21bada0b P d99276ea3dad479c9a077324c1e7dccc: Deleting WAL directory at /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-2/wal/wals/1040396622f846c5936ecc5b21bada0b
I20260808 16:55:12.262815 1975 raft_consensus.cc:2258] T 1040396622f846c5936ecc5b21bada0b P 275153aa50f44d19b906434156c28e31 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:55:12.263170 1974 ts_tablet_manager.cc:1939] T 1040396622f846c5936ecc5b21bada0b P d99276ea3dad479c9a077324c1e7dccc: Deleting consensus metadata
I20260808 16:55:12.263319 1975 ts_tablet_manager.cc:1905] T 1040396622f846c5936ecc5b21bada0b P 275153aa50f44d19b906434156c28e31: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:55:12.263654 1174 catalog_manager.cc:4841] TS d99276ea3dad479c9a077324c1e7dccc (127.8.24.3:33733): tablet 1040396622f846c5936ecc5b21bada0b (table test-table [id=15d36293befd42b8a42a3700a278ef3d]) successfully deleted
I20260808 16:55:12.264897 1975 ts_tablet_manager.cc:1918] T 1040396622f846c5936ecc5b21bada0b P 275153aa50f44d19b906434156c28e31: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 16:55:12.264988 1975 log.cc:1196] T 1040396622f846c5936ecc5b21bada0b P 275153aa50f44d19b906434156c28e31: Deleting WAL directory at /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-1/wal/wals/1040396622f846c5936ecc5b21bada0b
I20260808 16:55:12.265391 1975 ts_tablet_manager.cc:1939] T 1040396622f846c5936ecc5b21bada0b P 275153aa50f44d19b906434156c28e31: Deleting consensus metadata
I20260808 16:55:12.265908 1176 catalog_manager.cc:4841] TS 275153aa50f44d19b906434156c28e31 (127.8.24.2:44295): tablet 1040396622f846c5936ecc5b21bada0b (table test-table [id=15d36293befd42b8a42a3700a278ef3d]) successfully deleted
W20260808 16:55:12.269608 1174 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.24.1:37163 (error 108)
I20260808 16:55:12.269872 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 1514
I20260808 16:55:12.270205 1931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:12.270052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:37163 (local address 127.0.0.1:46264)
0808 16:55:12.270138 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:12.270170 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.1:37163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
W20260808 16:55:12.271513 1174 catalog_manager.cc:4584] TS 1d47cc8d97044f9aa2943ce89c4efaaa (127.8.24.1:37163): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1040396622f846c5936ecc5b21bada0b: Network error: Client connection negotiation failed: client connection to 127.8.24.1:37163: connect: Connection refused (error 111)
I20260808 16:55:12.276695 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 1729
I20260808 16:55:12.290524 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 1153
I20260808 16:55:32.303547 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:36933
--webserver_interface=127.8.24.62
--webserver_port=41471
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:36953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:36933
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:55:32.383852 1978 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:55:32.384029 1978 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:55:32.384049 1978 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:55:32.385098 1978 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 16:55:32.385130 1978 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 16:55:32.385142 1978 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 16:55:32.385152 1978 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 16:55:32.385169 1978 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:55:32.385179 1978 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:55:32.385196 1978 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 16:55:32.385213 1978 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 16:55:32.386597 1978 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:36953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:36933
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:36933
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.24.62
--webserver_port=41471
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1978
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:55:32.386773 1978 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:55:32.386940 1978 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:55:32.391522 1994 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:55:32.391577 1988 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:55:32.391633 1990 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:55:32.391649 1978 server_base.cc:1034] running on GCE node
I20260808 16:55:32.392026 1978 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:55:32.392299 1978 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:55:32.393440 1978 hybrid_clock.cc:648] HybridClock initialized: now 1786208132393427 us; error 28 us; skew 500 ppm
Aug 08 16:55:32 dist-test-slave-6zbq krb5kdc[316](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208132, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:55:32.395758 1978 init.cc:377] Logged in from keytab as kudu/127.8.24.62@KRBTEST.COM (short username kudu)
I20260808 16:55:32.396116 1978 webserver.cc:460] Webserver started at http://127.8.24.62:41471/ using document root <none> and password file <none>
I20260808 16:55:32.396299 1978 fs_manager.cc:359] Metadata directory not provided
I20260808 16:55:32.396345 1978 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:55:32.397478 1978 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 16:55:32.398036 2004 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:55:32.398180 1978 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 16:55:32.398249 1978 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/master-0/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/master-0/wal
uuid: "a623e9fb44f445dcb72293c81486a593"
format_stamp: "Formatted at 2026-08-08 16:54:50 on dist-test-slave-6zbq"
server_key: "f60301cb1fbbc542e658a7993d7d3aff"
server_key_iv: "9df75af9e4ade91284cc9b04c23adfa0"
server_key_version: "encryptionkey@0"
I20260808 16:55:32.398579 1978 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:55:32.409698 1978 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:55:32.411257 1978 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:55:32.411386 1978 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:55:32.414963 1978 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.62:36933
I20260808 16:55:32.415002 2076 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.62:36933 every 8 connection(s)
I20260808 16:55:32.415311 1978 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
I20260808 16:55:32.416604 2077 sys_catalog.cc:263] Verifying existing consensus state
I20260808 16:55:32.417214 2077 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a623e9fb44f445dcb72293c81486a593: Bootstrap starting.
I20260808 16:55:32.418699 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 1978
I20260808 16:55:32.418936 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.1:37163
--local_ip_for_outbound_sockets=127.8.24.1
--tserver_master_addrs=127.8.24.62:36933
--webserver_port=33579
--webserver_interface=127.8.24.1
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.1.keytab
--principal=kudu/127.8.24.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:36953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 16:55:32.425442 2077 log.cc:824] T 00000000000000000000000000000000 P a623e9fb44f445dcb72293c81486a593: Log is configured to *not* fsync() on all Append() calls
I20260808 16:55:32.427495 2077 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a623e9fb44f445dcb72293c81486a593: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 16:55:32.427739 2077 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a623e9fb44f445dcb72293c81486a593: Bootstrap complete.
I20260808 16:55:32.429642 2077 raft_consensus.cc:348] T 00000000000000000000000000000000 P a623e9fb44f445dcb72293c81486a593 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a623e9fb44f445dcb72293c81486a593" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 36933 } }
I20260808 16:55:32.429849 2077 raft_consensus.cc:729] T 00000000000000000000000000000000 P a623e9fb44f445dcb72293c81486a593 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: a623e9fb44f445dcb72293c81486a593, State: Initialized, Role: FOLLOWER
I20260808 16:55:32.429977 2077 consensus_queue.cc:260] T 00000000000000000000000000000000 P a623e9fb44f445dcb72293c81486a593 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a623e9fb44f445dcb72293c81486a593" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 36933 } }
I20260808 16:55:32.430032 2077 raft_consensus.cc:388] T 00000000000000000000000000000000 P a623e9fb44f445dcb72293c81486a593 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 16:55:32.430071 2077 raft_consensus.cc:482] T 00000000000000000000000000000000 P a623e9fb44f445dcb72293c81486a593 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 16:55:32.430114 2077 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a623e9fb44f445dcb72293c81486a593 [term 2 FOLLOWER]: Advancing to term 3
I20260808 16:55:32.431041 2077 raft_consensus.cc:504] T 00000000000000000000000000000000 P a623e9fb44f445dcb72293c81486a593 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a623e9fb44f445dcb72293c81486a593" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 36933 } }
I20260808 16:55:32.431128 2077 leader_election.cc:302] T 00000000000000000000000000000000 P a623e9fb44f445dcb72293c81486a593 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a623e9fb44f445dcb72293c81486a593; no voters:
I20260808 16:55:32.431289 2077 leader_election.cc:288] T 00000000000000000000000000000000 P a623e9fb44f445dcb72293c81486a593 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260808 16:55:32.431344 2082 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a623e9fb44f445dcb72293c81486a593 [term 3 FOLLOWER]: Leader election won for term 3
I20260808 16:55:32.431463 2082 raft_consensus.cc:686] T 00000000000000000000000000000000 P a623e9fb44f445dcb72293c81486a593 [term 3 LEADER]: Becoming Leader. State: Replica: a623e9fb44f445dcb72293c81486a593, State: Running, Role: LEADER
I20260808 16:55:32.431538 2077 sys_catalog.cc:564] T 00000000000000000000000000000000 P a623e9fb44f445dcb72293c81486a593 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 16:55:32.431608 2082 consensus_queue.cc:237] T 00000000000000000000000000000000 P a623e9fb44f445dcb72293c81486a593 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a623e9fb44f445dcb72293c81486a593" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 36933 } }
I20260808 16:55:32.431864 2082 sys_catalog.cc:455] T 00000000000000000000000000000000 P a623e9fb44f445dcb72293c81486a593 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a623e9fb44f445dcb72293c81486a593. Latest consensus state: current_term: 3 leader_uuid: "a623e9fb44f445dcb72293c81486a593" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a623e9fb44f445dcb72293c81486a593" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 36933 } } }
I20260808 16:55:32.432027 2086 sys_catalog.cc:455] T 00000000000000000000000000000000 P a623e9fb44f445dcb72293c81486a593 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "a623e9fb44f445dcb72293c81486a593" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a623e9fb44f445dcb72293c81486a593" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 36933 } } }
I20260808 16:55:32.432097 2082 sys_catalog.cc:458] T 00000000000000000000000000000000 P a623e9fb44f445dcb72293c81486a593 [sys.catalog]: This master's current role is: LEADER
I20260808 16:55:32.432193 2086 sys_catalog.cc:458] T 00000000000000000000000000000000 P a623e9fb44f445dcb72293c81486a593 [sys.catalog]: This master's current role is: LEADER
I20260808 16:55:32.432530 2096 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 16:55:32.433290 2096 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 16:55:32.433461 2096 catalog_manager.cc:1259] Loaded cluster ID: 92639f886e584e419174aa35f5ca4863
I20260808 16:55:32.433627 2096 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 16:55:32.434460 2096 catalog_manager.cc:1504] Loading token signing keys...
I20260808 16:55:32.434623 2096 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a623e9fb44f445dcb72293c81486a593: Loaded TSK: 0
I20260808 16:55:32.434682 2096 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a623e9fb44f445dcb72293c81486a593: Loaded TSK: 1
F20260808 16:55:32.434918 2096 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fa9087339ad google::LogMessage::Fail() at ??:0
@ 0x7fa90873688a google::LogMessage::SendToLog() at ??:0
@ 0x7fa9087334de google::LogMessage::Flush() at ??:0
@ 0x7fa908734a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fa90aea0104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7fa90ae9dd41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7fa91068d1cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7fa91068d48e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7fa91068d594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7fa910653c48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7fa91065dad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fa90a6010d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fa90a5f6954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fa90b45a6db start_thread at ??:0
@ 0x7fa90723161f clone at ??:0
W20260808 16:55:32.517153 2080 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:55:32.517323 2080 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:55:32.517343 2080 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:55:32.518455 2080 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:55:32.518494 2080 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:55:32.518519 2080 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.1
I20260808 16:55:32.519907 2080 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:36953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.1.keytab
--principal=kudu/127.8.24.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.1:37163
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.24.1
--webserver_port=33579
--tserver_master_addrs=127.8.24.62:36933
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2080
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.1
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:55:32.520092 2080 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:55:32.520292 2080 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:55:32.524755 2140 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:55:32.524751 2141 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:55:32.524797 2147 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:55:32.525123 2080 server_base.cc:1034] running on GCE node
I20260808 16:55:32.525298 2080 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:55:32.525501 2080 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:55:32.526636 2080 hybrid_clock.cc:648] HybridClock initialized: now 1786208132526624 us; error 33 us; skew 500 ppm
Aug 08 16:55:32 dist-test-slave-6zbq krb5kdc[316](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208132, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:55:32.529002 2080 init.cc:377] Logged in from keytab as kudu/127.8.24.1@KRBTEST.COM (short username kudu)
I20260808 16:55:32.529402 2080 webserver.cc:460] Webserver started at http://127.8.24.1:33579/ using document root <none> and password file <none>
I20260808 16:55:32.529603 2080 fs_manager.cc:359] Metadata directory not provided
I20260808 16:55:32.529655 2080 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:55:32.530886 2080 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 16:55:32.531469 2157 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:55:32.531651 2080 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 16:55:32.531750 2080 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-0/wal
uuid: "1d47cc8d97044f9aa2943ce89c4efaaa"
format_stamp: "Formatted at 2026-08-08 16:54:51 on dist-test-slave-6zbq"
server_key: "f2d1cdd77f20479ac7b19045512bb25d"
server_key_iv: "703017eb4f461fbc6dfd65dac530115c"
server_key_version: "encryptionkey@0"
I20260808 16:55:32.532116 2080 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:55:32.572836 2080 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:55:32.575083 2080 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:55:32.575232 2080 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:55:32.575464 2080 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:55:32.575891 2165 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 16:55:32.577240 2080 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 16:55:32.577298 2080 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260808 16:55:32.577330 2080 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 16:55:32.578154 2080 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 16:55:32.578199 2080 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260808 16:55:32.578239 2165 tablet_bootstrap.cc:492] T 1040396622f846c5936ecc5b21bada0b P 1d47cc8d97044f9aa2943ce89c4efaaa: Bootstrap starting.
I20260808 16:55:32.585155 2080 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.1:37163
I20260808 16:55:32.585536 2080 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-0/data/info.pb
I20260808 16:55:32.587917 2165 log.cc:824] T 1040396622f846c5936ecc5b21bada0b P 1d47cc8d97044f9aa2943ce89c4efaaa: Log is configured to *not* fsync() on all Append() calls
I20260808 16:55:32.588267 2343 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.1:37163 every 8 connection(s)
I20260808 16:55:32.589809 2165 tablet_bootstrap.cc:492] T 1040396622f846c5936ecc5b21bada0b P 1d47cc8d97044f9aa2943ce89c4efaaa: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 16:55:32.590061 2165 tablet_bootstrap.cc:492] T 1040396622f846c5936ecc5b21bada0b P 1d47cc8d97044f9aa2943ce89c4efaaa: Bootstrap complete.
I20260808 16:55:32.590835 2165 ts_tablet_manager.cc:1397] T 1040396622f846c5936ecc5b21bada0b P 1d47cc8d97044f9aa2943ce89c4efaaa: Time spent bootstrapping tablet: real 0.013s user 0.000s sys 0.012s
I20260808 16:55:32.591760 2165 raft_consensus.cc:348] T 1040396622f846c5936ecc5b21bada0b P 1d47cc8d97044f9aa2943ce89c4efaaa [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "275153aa50f44d19b906434156c28e31" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 44295 } } peers { permanent_uuid: "d99276ea3dad479c9a077324c1e7dccc" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 33733 } } peers { permanent_uuid: "1d47cc8d97044f9aa2943ce89c4efaaa" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 37163 } }
I20260808 16:55:32.591982 2165 raft_consensus.cc:729] T 1040396622f846c5936ecc5b21bada0b P 1d47cc8d97044f9aa2943ce89c4efaaa [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d47cc8d97044f9aa2943ce89c4efaaa, State: Initialized, Role: FOLLOWER
I20260808 16:55:32.592092 2165 consensus_queue.cc:260] T 1040396622f846c5936ecc5b21bada0b P 1d47cc8d97044f9aa2943ce89c4efaaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "275153aa50f44d19b906434156c28e31" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 44295 } } peers { permanent_uuid: "d99276ea3dad479c9a077324c1e7dccc" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 33733 } } peers { permanent_uuid: "1d47cc8d97044f9aa2943ce89c4efaaa" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 37163 } }
I20260808 16:55:32.592329 2165 ts_tablet_manager.cc:1428] T 1040396622f846c5936ecc5b21bada0b P 1d47cc8d97044f9aa2943ce89c4efaaa: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20260808 16:55:32.595042 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 2080
I20260808 16:55:32.595286 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.2:44295
--local_ip_for_outbound_sockets=127.8.24.2
--tserver_master_addrs=127.8.24.62:36933
--webserver_port=45671
--webserver_interface=127.8.24.2
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.2.keytab
--principal=kudu/127.8.24.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:36953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:55:32.633502 2348 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:55:32.586186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:36933 (local address 127.8.24.1:47525)
0808 16:55:32.586431 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:32.586485 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 16:55:32.587081 (+ 596us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:55:32.633355 (+ 46274us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:36933: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":96,"threads_started":1}
W20260808 16:55:32.633742 2344 heartbeater.cc:644] Failed to heartbeat to 127.8.24.62:36933 (0 consecutive failures): Network error: Failed to ping master at 127.8.24.62:36933: Client connection negotiation failed: client connection to 127.8.24.62:36933: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 16:55:32.634078 2348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:32.633974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:36933 (local address 127.8.24.1:51655)
0808 16:55:32.634019 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:32.634056 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260808 16:55:32.634418 2348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:32.634321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:36933 (local address 127.8.24.1:43993)
0808 16:55:32.634374 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:32.634398 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260808 16:55:32.634516 2344 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 16:55:32.680241 2354 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:55:32.680423 2354 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:55:32.680445 2354 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:55:32.681569 2354 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:55:32.681609 2354 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:55:32.681634 2354 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.2
I20260808 16:55:32.683063 2354 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:36953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.2.keytab
--principal=kudu/127.8.24.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.2:44295
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.24.2
--webserver_port=45671
--tserver_master_addrs=127.8.24.62:36933
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2354
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.2
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:55:32.683234 2354 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:55:32.683408 2354 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:55:32.687868 2359 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:55:32.687893 2362 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:55:32.687986 2354 server_base.cc:1034] running on GCE node
W20260808 16:55:32.688036 2360 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:55:32.688272 2354 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:55:32.688489 2354 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:55:32.689627 2354 hybrid_clock.cc:648] HybridClock initialized: now 1786208132689605 us; error 38 us; skew 500 ppm
Aug 08 16:55:32 dist-test-slave-6zbq krb5kdc[316](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208132, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:55:32.691800 2354 init.cc:377] Logged in from keytab as kudu/127.8.24.2@KRBTEST.COM (short username kudu)
I20260808 16:55:32.692185 2354 webserver.cc:460] Webserver started at http://127.8.24.2:45671/ using document root <none> and password file <none>
I20260808 16:55:32.692370 2354 fs_manager.cc:359] Metadata directory not provided
I20260808 16:55:32.692417 2354 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:55:32.693614 2354 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 16:55:32.694136 2373 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:55:32.694280 2354 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 16:55:32.694379 2354 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-1/wal
uuid: "275153aa50f44d19b906434156c28e31"
format_stamp: "Formatted at 2026-08-08 16:54:51 on dist-test-slave-6zbq"
server_key: "daa72f3463c2e38c9867de56b3e7cd98"
server_key_iv: "c7125c51b2b17114d39058e2334f00d9"
server_key_version: "encryptionkey@0"
I20260808 16:55:32.694662 2354 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:55:32.716354 2354 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:55:32.717861 2354 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:55:32.717983 2354 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:55:32.718182 2354 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:55:32.718474 2354 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:55:32.718508 2354 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:55:32.718535 2354 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:55:32.718559 2354 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:55:32.723881 2354 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.2:44295
I20260808 16:55:32.723938 2544 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.2:44295 every 8 connection(s)
I20260808 16:55:32.724223 2354 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-1/data/info.pb
I20260808 16:55:32.725093 2549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:32.724768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:36933 (local address 127.8.24.2:33487)
0808 16:55:32.724907 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:32.724968 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":75,"threads_started":1}
W20260808 16:55:32.725455 2546 heartbeater.cc:644] Failed to heartbeat to 127.8.24.62:36933 (0 consecutive failures): Network error: Failed to ping master at 127.8.24.62:36933: Client connection negotiation failed: client connection to 127.8.24.62:36933: connect: Connection refused (error 111)
I20260808 16:55:32.725696 2549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:32.725607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:36933 (local address 127.8.24.2:39201)
0808 16:55:32.725652 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:32.725676 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260808 16:55:32.725926 2549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:32.725855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:36933 (local address 127.8.24.2:51381)
0808 16:55:32.725887 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:32.725908 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260808 16:55:32.726011 2546 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 16:55:32.730798 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 2354
I20260808 16:55:32.731003 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.3:33733
--local_ip_for_outbound_sockets=127.8.24.3
--tserver_master_addrs=127.8.24.62:36933
--webserver_port=39907
--webserver_interface=127.8.24.3
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.3.keytab
--principal=kudu/127.8.24.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:36953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:55:32.809234 2550 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:55:32.809406 2550 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:55:32.809427 2550 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:55:32.810491 2550 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:55:32.810524 2550 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:55:32.810549 2550 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.3
I20260808 16:55:32.811913 2550 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:36953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.3.keytab
--principal=kudu/127.8.24.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.3:33733
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.24.3
--webserver_port=39907
--tserver_master_addrs=127.8.24.62:36933
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2550
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.3
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:55:32.812085 2550 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:55:32.812251 2550 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:55:32.816768 2561 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:55:32.816787 2550 server_base.cc:1034] running on GCE node
W20260808 16:55:32.816797 2560 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:55:32.816773 2563 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:55:32.817092 2550 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:55:32.817339 2550 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:55:32.818476 2550 hybrid_clock.cc:648] HybridClock initialized: now 1786208132818461 us; error 31 us; skew 500 ppm
Aug 08 16:55:32 dist-test-slave-6zbq krb5kdc[316](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208132, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:55:32.820706 2550 init.cc:377] Logged in from keytab as kudu/127.8.24.3@KRBTEST.COM (short username kudu)
I20260808 16:55:32.821134 2550 webserver.cc:460] Webserver started at http://127.8.24.3:39907/ using document root <none> and password file <none>
I20260808 16:55:32.821357 2550 fs_manager.cc:359] Metadata directory not provided
I20260808 16:55:32.821403 2550 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:55:32.822597 2550 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 16:55:32.823141 2577 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:55:32.823298 2550 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 16:55:32.823382 2550 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-2/wal
uuid: "d99276ea3dad479c9a077324c1e7dccc"
format_stamp: "Formatted at 2026-08-08 16:54:51 on dist-test-slave-6zbq"
server_key: "4f814eb022fcb67fa8f59f0e700aca0b"
server_key_iv: "4f681b16dacc81905593ce21969b24df"
server_key_version: "encryptionkey@0"
I20260808 16:55:32.823659 2550 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:55:32.859476 2550 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:55:32.861480 2550 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:55:32.861617 2550 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:55:32.861819 2550 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:55:32.862105 2550 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:55:32.862138 2550 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:55:32.862167 2550 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:55:32.862185 2550 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:55:32.867532 2550 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.3:33733
I20260808 16:55:32.867602 2751 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.3:33733 every 8 connection(s)
I20260808 16:55:32.867888 2550 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786207919384338-8288-0/minicluster-data/ts-2/data/info.pb
I20260808 16:55:32.868821 2756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:32.868478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:36933 (local address 127.8.24.3:44417)
0808 16:55:32.868613 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:32.868687 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":67,"threads_started":1}
W20260808 16:55:32.869205 2752 heartbeater.cc:644] Failed to heartbeat to 127.8.24.62:36933 (0 consecutive failures): Network error: Failed to ping master at 127.8.24.62:36933: Client connection negotiation failed: client connection to 127.8.24.62:36933: connect: Connection refused (error 111)
I20260808 16:55:32.869455 2756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:32.869375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:36933 (local address 127.8.24.3:38705)
0808 16:55:32.869411 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:32.869437 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260808 16:55:32.869670 2756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:32.869610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:36933 (local address 127.8.24.3:57893)
0808 16:55:32.869640 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:32.869657 (+ 17us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260808 16:55:32.869750 2752 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 16:55:32.876058 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 2550
I20260808 16:55:33.634975 2759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:33.634783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:36933 (local address 127.8.24.1:55939)
0808 16:55:33.634888 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:33.634915 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":57,"threads_started":1}
I20260808 16:55:33.726485 2760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:33.726300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:36933 (local address 127.8.24.2:44695)
0808 16:55:33.726407 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:33.726439 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":64,"threads_started":1}
I20260808 16:55:33.870195 2766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:33.870001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:36933 (local address 127.8.24.3:35799)
0808 16:55:33.870116 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:33.870148 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":67,"threads_started":1}
I20260808 16:55:33.956730 2767 raft_consensus.cc:482] T 1040396622f846c5936ecc5b21bada0b P 1d47cc8d97044f9aa2943ce89c4efaaa [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 16:55:33.956840 2767 raft_consensus.cc:504] T 1040396622f846c5936ecc5b21bada0b P 1d47cc8d97044f9aa2943ce89c4efaaa [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "275153aa50f44d19b906434156c28e31" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 44295 } } peers { permanent_uuid: "d99276ea3dad479c9a077324c1e7dccc" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 33733 } } peers { permanent_uuid: "1d47cc8d97044f9aa2943ce89c4efaaa" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 37163 } }
I20260808 16:55:33.957110 2767 leader_election.cc:288] T 1040396622f846c5936ecc5b21bada0b P 1d47cc8d97044f9aa2943ce89c4efaaa [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 275153aa50f44d19b906434156c28e31 (127.8.24.2:44295), d99276ea3dad479c9a077324c1e7dccc (127.8.24.3:33733)
Aug 08 16:55:33 dist-test-slave-6zbq krb5kdc[316](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208132, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.1@KRBTEST.COM for kudu/127.8.24.3@KRBTEST.COM
Aug 08 16:55:33 dist-test-slave-6zbq krb5kdc[316](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208132, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.1@KRBTEST.COM for kudu/127.8.24.2@KRBTEST.COM
I20260808 16:55:33.968348 2769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:33.957253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.3:33733 (local address 127.8.24.1:54381)
0808 16:55:33.957372 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:33.957382 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 16:55:33.957435 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:55:33.958392 (+ 957us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:55:33.958395 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:55:33.958853 (+ 458us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:55:33.959133 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:33.959141 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:33.960745 (+ 1604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:33.960746 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:33.962016 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:33.962019 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:33.962820 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:33.962821 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:33.962975 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:33.963535 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:55:33.963539 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:55:33.965108 (+ 1569us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:55:33.966793 (+ 1685us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:55:33.966800 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:55:33.966802 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:55:33.967091 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:55:33.967294 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:55:33.967295 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:55:33.967295 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:55:33.967424 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:55:33.967858 (+ 434us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:55:33.967864 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:55:33.967972 (+ 108us) client_negotiation.cc:769] Sending connection context
0808 16:55:33.968058 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 16:55:33.968220 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"mutex_wait_us":62,"thread_start_us":53,"threads_started":1}
I20260808 16:55:33.968365 2771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:33.957368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:54381 (local address 127.8.24.3:33733)
0808 16:55:33.957491 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0808 16:55:33.957503 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 16:55:33.957540 (+ 37us) server_negotiation.cc:407] Connection header received
0808 16:55:33.958096 (+ 556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:55:33.958100 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:55:33.958158 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:55:33.958305 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:55:33.959242 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:33.960639 (+ 1397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:33.962102 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:33.962743 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:33.962847 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:33.965219 (+ 2372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:55:33.965224 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:55:33.965224 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:55:33.965243 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:55:33.966712 (+ 1469us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:55:33.967163 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:55:33.967165 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:55:33.967166 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:55:33.967249 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:55:33.967490 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:55:33.967491 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:55:33.967491 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:55:33.967793 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:55:33.967873 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 16:55:33.968127 (+ 254us) server_negotiation.cc:299] Negotiation successful
0808 16:55:33.968262 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80,"thread_start_us":52,"threads_started":1}
I20260808 16:55:33.968468 2770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:33.957343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:58499 (local address 127.8.24.2:44295)
0808 16:55:33.957510 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0808 16:55:33.957516 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 16:55:33.957541 (+ 25us) server_negotiation.cc:407] Connection header received
0808 16:55:33.958097 (+ 556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:55:33.958100 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:55:33.958158 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:55:33.958307 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:55:33.959243 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:33.960639 (+ 1396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:33.962102 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:33.962733 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:33.962844 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:33.965468 (+ 2624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:55:33.965472 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:55:33.965472 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:55:33.965483 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:55:33.967165 (+ 1682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:55:33.967478 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:55:33.967481 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:55:33.967482 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:55:33.967586 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:55:33.967751 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:55:33.967752 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:55:33.967752 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:55:33.967992 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:55:33.968052 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 16:55:33.968258 (+ 206us) server_negotiation.cc:299] Negotiation successful
0808 16:55:33.968377 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":72,"threads_started":1}
I20260808 16:55:33.968353 2759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:33.957236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.2:44295 (local address 127.8.24.1:58499)
0808 16:55:33.957284 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:33.957295 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 16:55:33.957346 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:55:33.958392 (+ 1046us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:55:33.958395 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:55:33.958847 (+ 452us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:55:33.959133 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:33.959141 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:33.960744 (+ 1603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:33.960745 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:33.962016 (+ 1271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:33.962019 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:33.962823 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:33.962823 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:33.962975 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:33.963517 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:55:33.963528 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:55:33.965374 (+ 1846us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:55:33.967226 (+ 1852us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:55:33.967227 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:55:33.967228 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:55:33.967402 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:55:33.967650 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:55:33.967651 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:55:33.967651 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:55:33.967699 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:55:33.968054 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:55:33.968055 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:55:33.968092 (+ 37us) client_negotiation.cc:769] Sending connection context
0808 16:55:33.968155 (+ 63us) client_negotiation.cc:240] Negotiation successful
0808 16:55:33.968270 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32,"mutex_wait_us":114}
W20260808 16:55:33.969358 2159 leader_election.cc:341] T 1040396622f846c5936ecc5b21bada0b P 1d47cc8d97044f9aa2943ce89c4efaaa [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d99276ea3dad479c9a077324c1e7dccc (127.8.24.3:33733): Not found: Tablet not found: 1040396622f846c5936ecc5b21bada0b
W20260808 16:55:33.969446 2161 leader_election.cc:341] T 1040396622f846c5936ecc5b21bada0b P 1d47cc8d97044f9aa2943ce89c4efaaa [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 275153aa50f44d19b906434156c28e31 (127.8.24.2:44295): Not found: Tablet not found: 1040396622f846c5936ecc5b21bada0b
I20260808 16:55:33.969522 2161 leader_election.cc:302] T 1040396622f846c5936ecc5b21bada0b P 1d47cc8d97044f9aa2943ce89c4efaaa [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1d47cc8d97044f9aa2943ce89c4efaaa; no voters: 275153aa50f44d19b906434156c28e31, d99276ea3dad479c9a077324c1e7dccc
I20260808 16:55:33.969628 2767 raft_consensus.cc:2726] T 1040396622f846c5936ecc5b21bada0b P 1d47cc8d97044f9aa2943ce89c4efaaa [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 16:55:34.635546 2772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:34.635358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:36933 (local address 127.8.24.1:48459)
0808 16:55:34.635470 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:34.635503 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":62,"threads_started":1}
I20260808 16:55:34.726995 2774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:34.726839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:36933 (local address 127.8.24.2:49825)
0808 16:55:34.726931 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:34.726962 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79,"thread_start_us":50,"threads_started":1}
I20260808 16:55:34.870710 2775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:34.870538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:36933 (local address 127.8.24.3:38091)
0808 16:55:34.870643 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:34.870675 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":55,"threads_started":1}
I20260808 16:55:35.489706 2777 raft_consensus.cc:482] T 1040396622f846c5936ecc5b21bada0b P 1d47cc8d97044f9aa2943ce89c4efaaa [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 16:55:35.489794 2777 raft_consensus.cc:504] T 1040396622f846c5936ecc5b21bada0b P 1d47cc8d97044f9aa2943ce89c4efaaa [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "275153aa50f44d19b906434156c28e31" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 44295 } } peers { permanent_uuid: "d99276ea3dad479c9a077324c1e7dccc" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 33733 } } peers { permanent_uuid: "1d47cc8d97044f9aa2943ce89c4efaaa" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 37163 } }
I20260808 16:55:35.489956 2777 leader_election.cc:288] T 1040396622f846c5936ecc5b21bada0b P 1d47cc8d97044f9aa2943ce89c4efaaa [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 275153aa50f44d19b906434156c28e31 (127.8.24.2:44295), d99276ea3dad479c9a077324c1e7dccc (127.8.24.3:33733)
W20260808 16:55:35.490361 2161 leader_election.cc:341] T 1040396622f846c5936ecc5b21bada0b P 1d47cc8d97044f9aa2943ce89c4efaaa [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 275153aa50f44d19b906434156c28e31 (127.8.24.2:44295): Not found: Tablet not found: 1040396622f846c5936ecc5b21bada0b
W20260808 16:55:35.490446 2159 leader_election.cc:341] T 1040396622f846c5936ecc5b21bada0b P 1d47cc8d97044f9aa2943ce89c4efaaa [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d99276ea3dad479c9a077324c1e7dccc (127.8.24.3:33733): Not found: Tablet not found: 1040396622f846c5936ecc5b21bada0b
I20260808 16:55:35.490487 2159 leader_election.cc:302] T 1040396622f846c5936ecc5b21bada0b P 1d47cc8d97044f9aa2943ce89c4efaaa [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1d47cc8d97044f9aa2943ce89c4efaaa; no voters: 275153aa50f44d19b906434156c28e31, d99276ea3dad479c9a077324c1e7dccc
I20260808 16:55:35.490549 2777 raft_consensus.cc:2726] T 1040396622f846c5936ecc5b21bada0b P 1d47cc8d97044f9aa2943ce89c4efaaa [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 16:55:35.636089 2779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:35.635911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:36933 (local address 127.8.24.1:42991)
0808 16:55:35.636016 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:35.636048 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":56,"threads_started":1}
I20260808 16:55:35.727483 2781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:35.727313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:36933 (local address 127.8.24.2:49541)
0808 16:55:35.727418 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:35.727450 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":55,"threads_started":1}
I20260808 16:55:35.871263 2784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:35.871062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:36933 (local address 127.8.24.3:54823)
0808 16:55:35.871195 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:35.871228 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":75,"threads_started":1}
I20260808 16:55:36.636592 2786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:36.636417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:36933 (local address 127.8.24.1:59631)
0808 16:55:36.636519 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:36.636547 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":58,"threads_started":1}
I20260808 16:55:36.727988 2787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:36.727822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:36933 (local address 127.8.24.2:60651)
0808 16:55:36.727923 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:36.727953 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":60,"threads_started":1}
I20260808 16:55:36.871835 2788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:36.871633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:36933 (local address 127.8.24.3:50689)
0808 16:55:36.871760 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:36.871797 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":58,"threads_started":1}
I20260808 16:55:37.637130 2790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:37.636942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:36933 (local address 127.8.24.1:55587)
0808 16:55:37.637052 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:37.637085 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":59,"threads_started":1}
I20260808 16:55:37.728454 2791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:37.728286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:36933 (local address 127.8.24.2:45171)
0808 16:55:37.728388 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:37.728420 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":58,"threads_started":1}
I20260808 16:55:37.872375 2792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:37.872193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:36933 (local address 127.8.24.3:55375)
0808 16:55:37.872300 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:37.872331 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":52,"threads_started":1}
I20260808 16:55:38.455005 2795 raft_consensus.cc:482] T 1040396622f846c5936ecc5b21bada0b P 1d47cc8d97044f9aa2943ce89c4efaaa [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 16:55:38.455092 2795 raft_consensus.cc:504] T 1040396622f846c5936ecc5b21bada0b P 1d47cc8d97044f9aa2943ce89c4efaaa [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "275153aa50f44d19b906434156c28e31" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 44295 } } peers { permanent_uuid: "d99276ea3dad479c9a077324c1e7dccc" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 33733 } } peers { permanent_uuid: "1d47cc8d97044f9aa2943ce89c4efaaa" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 37163 } }
I20260808 16:55:38.455264 2795 leader_election.cc:288] T 1040396622f846c5936ecc5b21bada0b P 1d47cc8d97044f9aa2943ce89c4efaaa [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 275153aa50f44d19b906434156c28e31 (127.8.24.2:44295), d99276ea3dad479c9a077324c1e7dccc (127.8.24.3:33733)
W20260808 16:55:38.455567 2161 leader_election.cc:341] T 1040396622f846c5936ecc5b21bada0b P 1d47cc8d97044f9aa2943ce89c4efaaa [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 275153aa50f44d19b906434156c28e31 (127.8.24.2:44295): Not found: Tablet not found: 1040396622f846c5936ecc5b21bada0b
W20260808 16:55:38.455651 2159 leader_election.cc:341] T 1040396622f846c5936ecc5b21bada0b P 1d47cc8d97044f9aa2943ce89c4efaaa [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d99276ea3dad479c9a077324c1e7dccc (127.8.24.3:33733): Not found: Tablet not found: 1040396622f846c5936ecc5b21bada0b
I20260808 16:55:38.455722 2159 leader_election.cc:302] T 1040396622f846c5936ecc5b21bada0b P 1d47cc8d97044f9aa2943ce89c4efaaa [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1d47cc8d97044f9aa2943ce89c4efaaa; no voters: 275153aa50f44d19b906434156c28e31, d99276ea3dad479c9a077324c1e7dccc
I20260808 16:55:38.455799 2795 raft_consensus.cc:2726] T 1040396622f846c5936ecc5b21bada0b P 1d47cc8d97044f9aa2943ce89c4efaaa [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 16:55:38.637645 2796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:38.637461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:36933 (local address 127.8.24.1:39813)
0808 16:55:38.637566 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:38.637597 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":58,"threads_started":1}
I20260808 16:55:38.728969 2798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:38.728791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:36933 (local address 127.8.24.2:47159)
0808 16:55:38.728900 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:38.728931 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":60,"threads_started":1}
I20260808 16:55:38.872967 2800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:38.872760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:36933 (local address 127.8.24.3:33545)
0808 16:55:38.872885 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:38.872925 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":60,"threads_started":1}
I20260808 16:55:39.638201 2801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:39.638011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:36933 (local address 127.8.24.1:33783)
0808 16:55:39.638118 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:39.638150 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":58,"threads_started":1}
I20260808 16:55:39.729486 2802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:39.729317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:36933 (local address 127.8.24.2:33489)
0808 16:55:39.729419 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:39.729450 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":60,"threads_started":1}
I20260808 16:55:39.873529 2806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:39.873309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:36933 (local address 127.8.24.3:59563)
0808 16:55:39.873460 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:39.873494 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":85,"threads_started":1}
I20260808 16:55:40.638749 2807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:40.638511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:36933 (local address 127.8.24.1:47133)
0808 16:55:40.638645 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:40.638702 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":78,"threads_started":1}
I20260808 16:55:40.730057 2809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:40.729836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:36933 (local address 127.8.24.2:57643)
0808 16:55:40.729967 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:40.730004 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":82,"threads_started":1}
I20260808 16:55:40.874111 2812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:40.873908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:36933 (local address 127.8.24.3:53511)
0808 16:55:40.874041 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:40.874074 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":76,"threads_started":1}
I20260808 16:55:41.446282 2813 raft_consensus.cc:482] T 1040396622f846c5936ecc5b21bada0b P 1d47cc8d97044f9aa2943ce89c4efaaa [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 16:55:41.446401 2813 raft_consensus.cc:504] T 1040396622f846c5936ecc5b21bada0b P 1d47cc8d97044f9aa2943ce89c4efaaa [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "275153aa50f44d19b906434156c28e31" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 44295 } } peers { permanent_uuid: "d99276ea3dad479c9a077324c1e7dccc" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 33733 } } peers { permanent_uuid: "1d47cc8d97044f9aa2943ce89c4efaaa" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 37163 } }
I20260808 16:55:41.446574 2813 leader_election.cc:288] T 1040396622f846c5936ecc5b21bada0b P 1d47cc8d97044f9aa2943ce89c4efaaa [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 275153aa50f44d19b906434156c28e31 (127.8.24.2:44295), d99276ea3dad479c9a077324c1e7dccc (127.8.24.3:33733)
W20260808 16:55:41.446924 2159 leader_election.cc:341] T 1040396622f846c5936ecc5b21bada0b P 1d47cc8d97044f9aa2943ce89c4efaaa [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d99276ea3dad479c9a077324c1e7dccc (127.8.24.3:33733): Not found: Tablet not found: 1040396622f846c5936ecc5b21bada0b
W20260808 16:55:41.447016 2161 leader_election.cc:341] T 1040396622f846c5936ecc5b21bada0b P 1d47cc8d97044f9aa2943ce89c4efaaa [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 275153aa50f44d19b906434156c28e31 (127.8.24.2:44295): Not found: Tablet not found: 1040396622f846c5936ecc5b21bada0b
I20260808 16:55:41.447059 2161 leader_election.cc:302] T 1040396622f846c5936ecc5b21bada0b P 1d47cc8d97044f9aa2943ce89c4efaaa [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1d47cc8d97044f9aa2943ce89c4efaaa; no voters: 275153aa50f44d19b906434156c28e31, d99276ea3dad479c9a077324c1e7dccc
I20260808 16:55:41.447149 2813 raft_consensus.cc:2726] T 1040396622f846c5936ecc5b21bada0b P 1d47cc8d97044f9aa2943ce89c4efaaa [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 16:55:41.639317 2815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:41.639109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:36933 (local address 127.8.24.1:43929)
0808 16:55:41.639238 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:41.639272 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":62,"threads_started":1}
I20260808 16:55:41.730633 2818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:41.730423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:36933 (local address 127.8.24.2:45839)
0808 16:55:41.730544 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:41.730579 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":74,"threads_started":1}
I20260808 16:55:41.874708 2820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:41.874508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:36933 (local address 127.8.24.3:44255)
0808 16:55:41.874623 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:41.874666 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":58,"threads_started":1}
I20260808 16:55:42.639878 2821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:42.639672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:36933 (local address 127.8.24.1:40333)
0808 16:55:42.639797 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:42.639831 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":60,"threads_started":1}
I20260808 16:55:42.731253 2824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:42.730990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:36933 (local address 127.8.24.2:35645)
0808 16:55:42.731176 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:42.731215 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":62,"threads_started":1}
I20260808 16:55:42.875283 2825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:42.875082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:36933 (local address 127.8.24.3:54849)
0808 16:55:42.875208 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:42.875242 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":74,"threads_started":1}
I20260808 16:55:43.640391 2827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:43.640217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:36933 (local address 127.8.24.1:59937)
0808 16:55:43.640320 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:43.640348 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":55,"threads_started":1}
I20260808 16:55:43.732803 2828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:43.731619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:36933 (local address 127.8.24.2:53969)
0808 16:55:43.732738 (+ 1119us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:43.732767 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1107,"thread_start_us":1079,"threads_started":1}
I20260808 16:55:43.875792 2829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:43.875623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:36933 (local address 127.8.24.3:40563)
0808 16:55:43.875723 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:43.875756 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86,"thread_start_us":62,"threads_started":1}
I20260808 16:55:44.640914 2831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:44.640730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:36933 (local address 127.8.24.1:52555)
0808 16:55:44.640839 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:44.640872 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":60,"threads_started":1}
I20260808 16:55:44.733326 2832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:44.733149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:36933 (local address 127.8.24.2:38177)
0808 16:55:44.733256 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:44.733289 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":57,"threads_started":1}
I20260808 16:55:44.876291 2836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:44.876119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:36933 (local address 127.8.24.3:43571)
0808 16:55:44.876223 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:44.876255 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":56,"threads_started":1}
I20260808 16:55:45.641456 2837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:45.641278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:36933 (local address 127.8.24.1:48995)
0808 16:55:45.641382 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:45.641413 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":61,"threads_started":1}
I20260808 16:55:45.733861 2838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:45.733679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:36933 (local address 127.8.24.2:45511)
0808 16:55:45.733793 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:45.733824 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":59,"threads_started":1}
I20260808 16:55:45.876843 2839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:45.876650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:36933 (local address 127.8.24.3:53149)
0808 16:55:45.876777 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:45.876808 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":75,"threads_started":1}
I20260808 16:55:46.641988 2841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:46.641800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:36933 (local address 127.8.24.1:40217)
0808 16:55:46.641914 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:46.641943 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":60,"threads_started":1}
I20260808 16:55:46.734447 2842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:46.734232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:36933 (local address 127.8.24.2:58383)
0808 16:55:46.734368 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:46.734407 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":70,"threads_started":1}
I20260808 16:55:46.877416 2843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:46.877225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:36933 (local address 127.8.24.3:56283)
0808 16:55:46.877349 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:46.877380 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":76,"threads_started":1}
I20260808 16:55:46.942325 2845 raft_consensus.cc:482] T 1040396622f846c5936ecc5b21bada0b P 1d47cc8d97044f9aa2943ce89c4efaaa [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 16:55:46.942445 2845 raft_consensus.cc:504] T 1040396622f846c5936ecc5b21bada0b P 1d47cc8d97044f9aa2943ce89c4efaaa [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "275153aa50f44d19b906434156c28e31" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 44295 } } peers { permanent_uuid: "d99276ea3dad479c9a077324c1e7dccc" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 33733 } } peers { permanent_uuid: "1d47cc8d97044f9aa2943ce89c4efaaa" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 37163 } }
I20260808 16:55:46.942581 2845 leader_election.cc:288] T 1040396622f846c5936ecc5b21bada0b P 1d47cc8d97044f9aa2943ce89c4efaaa [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 275153aa50f44d19b906434156c28e31 (127.8.24.2:44295), d99276ea3dad479c9a077324c1e7dccc (127.8.24.3:33733)
W20260808 16:55:46.942921 2161 leader_election.cc:341] T 1040396622f846c5936ecc5b21bada0b P 1d47cc8d97044f9aa2943ce89c4efaaa [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 275153aa50f44d19b906434156c28e31 (127.8.24.2:44295): Not found: Tablet not found: 1040396622f846c5936ecc5b21bada0b
W20260808 16:55:46.942996 2159 leader_election.cc:341] T 1040396622f846c5936ecc5b21bada0b P 1d47cc8d97044f9aa2943ce89c4efaaa [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d99276ea3dad479c9a077324c1e7dccc (127.8.24.3:33733): Not found: Tablet not found: 1040396622f846c5936ecc5b21bada0b
I20260808 16:55:46.943045 2159 leader_election.cc:302] T 1040396622f846c5936ecc5b21bada0b P 1d47cc8d97044f9aa2943ce89c4efaaa [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1d47cc8d97044f9aa2943ce89c4efaaa; no voters: 275153aa50f44d19b906434156c28e31, d99276ea3dad479c9a077324c1e7dccc
I20260808 16:55:46.943112 2845 raft_consensus.cc:2726] T 1040396622f846c5936ecc5b21bada0b P 1d47cc8d97044f9aa2943ce89c4efaaa [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 16:55:47.642601 2848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:47.642418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:36933 (local address 127.8.24.1:60185)
0808 16:55:47.642526 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:47.642557 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":58,"threads_started":1}
I20260808 16:55:47.735041 2851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:47.734867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:36933 (local address 127.8.24.2:37833)
0808 16:55:47.734972 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:47.735003 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":56,"threads_started":1}
I20260808 16:55:47.877970 2852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:47.877787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:36933 (local address 127.8.24.3:56157)
0808 16:55:47.877906 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:47.877936 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":76,"threads_started":1}
I20260808 16:55:47.882015 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 2080
I20260808 16:55:47.898648 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 2354
I20260808 16:55:47.908151 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 2550
2026-08-08T16:55:47Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58379 ms)
[----------] 30 tests from SecurityITest (228519 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786207919384338-8288-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 16:55:47 dist-test-slave-6zbq krb5kdc[2856](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 16:55:47 dist-test-slave-6zbq krb5kdc[2856](info): set up 2 sockets
Aug 08 16:55:47 dist-test-slave-6zbq krb5kdc[2856](info): commencing operation
krb5kdc: starting...
W20260808 16:55:49.042506 8288 mini_kdc.cc:121] Time spent starting KDC: real 1.112s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786207919384338-8288-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786207919384338-8288-0/krb5kdc/test-user.keytab.
Aug 08 16:55:49 dist-test-slave-6zbq krb5kdc[2856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208149, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T16:55:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T16:55:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.24.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
I20260808 16:55:49.147261 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786207919384338-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786207919384338-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786207919384338-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:38177
--webserver_interface=127.8.24.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:39795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:38177
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:55:49.226281 2882 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:55:49.226480 2882 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:55:49.226511 2882 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:55:49.227603 2882 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 16:55:49.227643 2882 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:55:49.227661 2882 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:55:49.227679 2882 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 16:55:49.227694 2882 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 16:55:49.229086 2882 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:39795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786207919384338-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786207919384338-8288-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:38177
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:38177
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.24.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2882
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786207919384338-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:55:49.229318 2882 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:55:49.229507 2882 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:55:49.233784 2893 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:55:49.233789 2897 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:55:49.233798 2894 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:55:49.234030 2882 server_base.cc:1034] running on GCE node
I20260808 16:55:49.234268 2882 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:55:49.234550 2882 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:55:49.235687 2882 hybrid_clock.cc:648] HybridClock initialized: now 1786208149235672 us; error 27 us; skew 500 ppm
Aug 08 16:55:49 dist-test-slave-6zbq krb5kdc[2856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208149, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:55:49.241003 2882 init.cc:377] Logged in from keytab as kudu/127.8.24.62@KRBTEST.COM (short username kudu)
I20260808 16:55:49.241365 2882 webserver.cc:460] Webserver started at http://127.8.24.62:40937/ using document root <none> and password file <none>
I20260808 16:55:49.241523 2882 fs_manager.cc:359] Metadata directory not provided
I20260808 16:55:49.241559 2882 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:55:49.241642 2882 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:55:49.242600 2882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786207919384338-8288-0/minicluster-data/master-0/data/instance:
uuid: "e8b2cffc97a04532abae36dfc65ce170"
format_stamp: "Formatted at 2026-08-08 16:55:49 on dist-test-slave-6zbq"
server_key: "270a36efc77e24a0946b777b592abbb7"
server_key_iv: "16ac72ee4dcbbd5852b4966349647d44"
server_key_version: "encryptionkey@0"
I20260808 16:55:49.242892 2882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786207919384338-8288-0/minicluster-data/master-0/wal/instance:
uuid: "e8b2cffc97a04532abae36dfc65ce170"
format_stamp: "Formatted at 2026-08-08 16:55:49 on dist-test-slave-6zbq"
server_key: "270a36efc77e24a0946b777b592abbb7"
server_key_iv: "16ac72ee4dcbbd5852b4966349647d44"
server_key_version: "encryptionkey@0"
I20260808 16:55:49.243999 2882 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 16:55:49.244582 2909 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:55:49.244727 2882 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 16:55:49.244807 2882 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786207919384338-8288-0/minicluster-data/master-0/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786207919384338-8288-0/minicluster-data/master-0/wal
uuid: "e8b2cffc97a04532abae36dfc65ce170"
format_stamp: "Formatted at 2026-08-08 16:55:49 on dist-test-slave-6zbq"
server_key: "270a36efc77e24a0946b777b592abbb7"
server_key_iv: "16ac72ee4dcbbd5852b4966349647d44"
server_key_version: "encryptionkey@0"
I20260808 16:55:49.244863 2882 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786207919384338-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786207919384338-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786207919384338-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:55:49.258927 2882 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:55:49.260454 2882 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:55:49.260569 2882 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:55:49.264226 2882 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.62:38177
I20260808 16:55:49.264254 2993 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.62:38177 every 8 connection(s)
I20260808 16:55:49.264528 2882 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
I20260808 16:55:49.265293 2994 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:55:49.267313 2994 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8b2cffc97a04532abae36dfc65ce170: Bootstrap starting.
I20260808 16:55:49.267791 2994 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e8b2cffc97a04532abae36dfc65ce170: Neither blocks nor log segments found. Creating new log.
I20260808 16:55:49.268004 2994 log.cc:824] T 00000000000000000000000000000000 P e8b2cffc97a04532abae36dfc65ce170: Log is configured to *not* fsync() on all Append() calls
I20260808 16:55:49.268589 2994 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8b2cffc97a04532abae36dfc65ce170: No bootstrap required, opened a new log
I20260808 16:55:49.270042 2994 raft_consensus.cc:348] T 00000000000000000000000000000000 P e8b2cffc97a04532abae36dfc65ce170 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8b2cffc97a04532abae36dfc65ce170" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 38177 } }
I20260808 16:55:49.270162 2994 raft_consensus.cc:374] T 00000000000000000000000000000000 P e8b2cffc97a04532abae36dfc65ce170 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:55:49.270183 2994 raft_consensus.cc:729] T 00000000000000000000000000000000 P e8b2cffc97a04532abae36dfc65ce170 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8b2cffc97a04532abae36dfc65ce170, State: Initialized, Role: FOLLOWER
I20260808 16:55:49.270296 2994 consensus_queue.cc:260] T 00000000000000000000000000000000 P e8b2cffc97a04532abae36dfc65ce170 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8b2cffc97a04532abae36dfc65ce170" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 38177 } }
I20260808 16:55:49.270395 2994 raft_consensus.cc:388] T 00000000000000000000000000000000 P e8b2cffc97a04532abae36dfc65ce170 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 16:55:49.270426 2994 raft_consensus.cc:482] T 00000000000000000000000000000000 P e8b2cffc97a04532abae36dfc65ce170 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 16:55:49.270457 2994 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e8b2cffc97a04532abae36dfc65ce170 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:55:49.270885 2994 raft_consensus.cc:504] T 00000000000000000000000000000000 P e8b2cffc97a04532abae36dfc65ce170 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8b2cffc97a04532abae36dfc65ce170" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 38177 } }
I20260808 16:55:49.270967 2994 leader_election.cc:302] T 00000000000000000000000000000000 P e8b2cffc97a04532abae36dfc65ce170 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e8b2cffc97a04532abae36dfc65ce170; no voters:
I20260808 16:55:49.271088 2994 leader_election.cc:288] T 00000000000000000000000000000000 P e8b2cffc97a04532abae36dfc65ce170 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 16:55:49.271138 2999 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e8b2cffc97a04532abae36dfc65ce170 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:55:49.271201 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 2882
I20260808 16:55:49.271281 2999 raft_consensus.cc:686] T 00000000000000000000000000000000 P e8b2cffc97a04532abae36dfc65ce170 [term 1 LEADER]: Becoming Leader. State: Replica: e8b2cffc97a04532abae36dfc65ce170, State: Running, Role: LEADER
I20260808 16:55:49.271303 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786207919384338-8288-0/minicluster-data/master-0/wal/instance
I20260808 16:55:49.271338 2994 sys_catalog.cc:564] T 00000000000000000000000000000000 P e8b2cffc97a04532abae36dfc65ce170 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 16:55:49.271373 2999 consensus_queue.cc:237] T 00000000000000000000000000000000 P e8b2cffc97a04532abae36dfc65ce170 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8b2cffc97a04532abae36dfc65ce170" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 38177 } }
I20260808 16:55:49.271494 8288 external_mini_cluster.cc:1428] Setting key 0d201cc5ed540e8abe415d517300919d
I20260808 16:55:49.271656 2999 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8b2cffc97a04532abae36dfc65ce170 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e8b2cffc97a04532abae36dfc65ce170" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8b2cffc97a04532abae36dfc65ce170" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 38177 } } }
I20260808 16:55:49.271831 2999 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8b2cffc97a04532abae36dfc65ce170 [sys.catalog]: This master's current role is: LEADER
I20260808 16:55:49.272014 2999 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8b2cffc97a04532abae36dfc65ce170 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e8b2cffc97a04532abae36dfc65ce170. Latest consensus state: current_term: 1 leader_uuid: "e8b2cffc97a04532abae36dfc65ce170" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8b2cffc97a04532abae36dfc65ce170" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 38177 } } }
I20260808 16:55:49.272079 2999 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8b2cffc97a04532abae36dfc65ce170 [sys.catalog]: This master's current role is: LEADER
I20260808 16:55:49.272365 3013 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 16:55:49.272876 3013 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 16:55:49.274420 3013 catalog_manager.cc:1347] Generated new cluster ID: d90838ec7950402797126ac6c8c3de56
I20260808 16:55:49.274467 3013 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 16:55:49 dist-test-slave-6zbq krb5kdc[2856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208149, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260808 16:55:49.282096 3013 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 16:55:49.282691 3013 catalog_manager.cc:1504] Loading token signing keys...
I20260808 16:55:49.283052 3015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:49.272034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49664 (local address 127.8.24.62:38177)
0808 16:55:49.272392 (+ 358us) server_negotiation.cc:206] Beginning negotiation
0808 16:55:49.272401 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 16:55:49.272438 (+ 37us) server_negotiation.cc:407] Connection header received
0808 16:55:49.272548 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:55:49.272554 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:55:49.272607 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:55:49.272728 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:55:49.273531 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:49.274725 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:49.275799 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:49.276383 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:49.276468 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:49.279022 (+ 2554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:55:49.279026 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:55:49.279027 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:55:49.279038 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:55:49.281443 (+ 2405us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:55:49.281822 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:55:49.281824 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:55:49.281824 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:55:49.281926 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:55:49.282133 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:55:49.282134 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:55:49.282134 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:55:49.282442 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:55:49.282593 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0808 16:55:49.282688 (+ 95us) server_negotiation.cc:299] Negotiation successful
0808 16:55:49.282838 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":79,"threads_started":1}
I20260808 16:55:49.291399 3013 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e8b2cffc97a04532abae36dfc65ce170: Generated new TSK 0
I20260808 16:55:49.291539 3013 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.24.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.1.keytab.
Entry for principal kudu/127.8.24.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.1.keytab.
I20260808 16:55:49.319175 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786207919384338-8288-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786207919384338-8288-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786207919384338-8288-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786207919384338-8288-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.1:0
--local_ip_for_outbound_sockets=127.8.24.1
--webserver_interface=127.8.24.1
--webserver_port=0
--tserver_master_addrs=127.8.24.62:38177
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.1.keytab
--principal=kudu/127.8.24.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:39795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:55:49.396724 3031 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:55:49.396908 3031 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:55:49.396930 3031 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:55:49.397976 3031 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:55:49.398007 3031 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:55:49.398032 3031 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.1
I20260808 16:55:49.399389 3031 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:39795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786207919384338-8288-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786207919384338-8288-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.1.keytab
--principal=kudu/127.8.24.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786207919384338-8288-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.24.1
--webserver_port=0
--tserver_master_addrs=127.8.24.62:38177
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3031
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.1
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786207919384338-8288-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:55:49.399555 3031 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:55:49.399719 3031 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:55:49.404166 3038 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:55:49.404206 3031 server_base.cc:1034] running on GCE node
W20260808 16:55:49.404244 3037 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:55:49.404160 3041 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:55:49.404567 3031 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:55:49.404769 3031 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:55:49.405897 3031 hybrid_clock.cc:648] HybridClock initialized: now 1786208149405887 us; error 27 us; skew 500 ppm
Aug 08 16:55:49 dist-test-slave-6zbq krb5kdc[2856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208149, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:55:49.408183 3031 init.cc:377] Logged in from keytab as kudu/127.8.24.1@KRBTEST.COM (short username kudu)
I20260808 16:55:49.408582 3031 webserver.cc:460] Webserver started at http://127.8.24.1:38831/ using document root <none> and password file <none>
I20260808 16:55:49.408764 3031 fs_manager.cc:359] Metadata directory not provided
I20260808 16:55:49.408815 3031 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:55:49.408936 3031 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:55:49.409847 3031 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786207919384338-8288-0/minicluster-data/ts-0/data/instance:
uuid: "05caaa0ef0484dc8bae1c3dede53c76a"
format_stamp: "Formatted at 2026-08-08 16:55:49 on dist-test-slave-6zbq"
server_key: "136b8655cdd9c95d1b2de8b2ad100288"
server_key_iv: "46b72180e2aea99c4ca078d19daa0927"
server_key_version: "encryptionkey@0"
I20260808 16:55:49.410156 3031 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786207919384338-8288-0/minicluster-data/ts-0/wal/instance:
uuid: "05caaa0ef0484dc8bae1c3dede53c76a"
format_stamp: "Formatted at 2026-08-08 16:55:49 on dist-test-slave-6zbq"
server_key: "136b8655cdd9c95d1b2de8b2ad100288"
server_key_iv: "46b72180e2aea99c4ca078d19daa0927"
server_key_version: "encryptionkey@0"
I20260808 16:55:49.411360 3031 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260808 16:55:49.411984 3051 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:55:49.412123 3031 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 16:55:49.412201 3031 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786207919384338-8288-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786207919384338-8288-0/minicluster-data/ts-0/wal
uuid: "05caaa0ef0484dc8bae1c3dede53c76a"
format_stamp: "Formatted at 2026-08-08 16:55:49 on dist-test-slave-6zbq"
server_key: "136b8655cdd9c95d1b2de8b2ad100288"
server_key_iv: "46b72180e2aea99c4ca078d19daa0927"
server_key_version: "encryptionkey@0"
I20260808 16:55:49.412266 3031 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786207919384338-8288-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786207919384338-8288-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786207919384338-8288-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:55:49.426261 3031 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:55:49.427788 3031 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:55:49.427906 3031 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:55:49.428087 3031 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:55:49.428339 3031 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:55:49.428370 3031 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:55:49.428396 3031 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:55:49.428414 3031 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:55:49.433681 3031 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.1:46153
I20260808 16:55:49.433724 3229 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.1:46153 every 8 connection(s)
I20260808 16:55:49.433980 3031 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786207919384338-8288-0/minicluster-data/ts-0/data/info.pb
Aug 08 16:55:49 dist-test-slave-6zbq krb5kdc[2856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208149, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.1@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260808 16:55:49.441428 3015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:49.434596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:49227 (local address 127.8.24.62:38177)
0808 16:55:49.434652 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 16:55:49.434654 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:55:49.434848 (+ 194us) server_negotiation.cc:407] Connection header received
0808 16:55:49.435237 (+ 389us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:55:49.435238 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:55:49.435246 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:55:49.435295 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:55:49.436067 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:49.436536 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:49.437396 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:49.437573 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:49.437616 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:49.439420 (+ 1804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:55:49.439421 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:55:49.439422 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:55:49.439428 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:55:49.440437 (+ 1009us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:55:49.440761 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:55:49.440761 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:55:49.440762 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:55:49.440806 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:55:49.440975 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:55:49.440975 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:55:49.440975 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:55:49.441068 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:55:49.441126 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 16:55:49.441311 (+ 185us) server_negotiation.cc:299] Negotiation successful
0808 16:55:49.441370 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 16:55:49.441558 3232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:49.434574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:38177 (local address 127.8.24.1:49227)
0808 16:55:49.434726 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:49.434763 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 16:55:49.435159 (+ 396us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:55:49.435343 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:55:49.435346 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:55:49.435840 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:55:49.436002 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:49.436006 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:49.436595 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:49.436596 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:49.437332 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:49.437334 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:49.437624 (+ 290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:49.437625 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:49.437707 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:49.438047 (+ 340us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:55:49.438054 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:55:49.439341 (+ 1287us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:55:49.440499 (+ 1158us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:55:49.440505 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:55:49.440506 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:55:49.440708 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:55:49.440847 (+ 139us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:55:49.440848 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:55:49.440848 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:55:49.440930 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:55:49.441119 (+ 189us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:55:49.441122 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:55:49.441217 (+ 95us) client_negotiation.cc:769] Sending connection context
0808 16:55:49.441287 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 16:55:49.441390 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":76,"threads_started":1}
I20260808 16:55:49.442036 3230 heartbeater.cc:347] Connected to a master server at 127.8.24.62:38177
I20260808 16:55:49.442114 3230 heartbeater.cc:464] Registering TS with master...
I20260808 16:55:49.442269 3230 heartbeater.cc:511] Master 127.8.24.62:38177 requested a full tablet report, sending...
I20260808 16:55:49.442732 2938 ts_manager.cc:194] Registered new tserver with Master: 05caaa0ef0484dc8bae1c3dede53c76a (127.8.24.1:46153)
I20260808 16:55:49.443167 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 3031
I20260808 16:55:49.443270 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786207919384338-8288-0/minicluster-data/ts-0/wal/instance
I20260808 16:55:49.443365 2938 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.1@KRBTEST.COM'} at 127.8.24.1:49227
I20260808 16:55:49.443434 8288 external_mini_cluster.cc:1428] Setting key 3941ac7fe7f3e3773107c298873a28a2
WARNING: no policy specified for kudu/127.8.24.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.2.keytab.
Entry for principal kudu/127.8.24.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.2.keytab.
I20260808 16:55:49.464717 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786207919384338-8288-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786207919384338-8288-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786207919384338-8288-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786207919384338-8288-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.2:0
--local_ip_for_outbound_sockets=127.8.24.2
--webserver_interface=127.8.24.2
--webserver_port=0
--tserver_master_addrs=127.8.24.62:38177
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.2.keytab
--principal=kudu/127.8.24.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:39795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:55:49.541257 3237 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:55:49.541433 3237 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:55:49.541463 3237 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:55:49.542567 3237 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:55:49.542603 3237 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:55:49.542637 3237 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.2
I20260808 16:55:49.544019 3237 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:39795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786207919384338-8288-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786207919384338-8288-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.2.keytab
--principal=kudu/127.8.24.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786207919384338-8288-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.24.2
--webserver_port=0
--tserver_master_addrs=127.8.24.62:38177
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3237
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.2
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786207919384338-8288-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:55:49.544232 3237 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:55:49.544430 3237 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:55:49.548811 3244 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:55:49.548811 3242 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:55:49.548821 3246 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:55:49.549010 3237 server_base.cc:1034] running on GCE node
I20260808 16:55:49.549230 3237 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:55:49.549455 3237 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:55:49.550586 3237 hybrid_clock.cc:648] HybridClock initialized: now 1786208149550571 us; error 32 us; skew 500 ppm
Aug 08 16:55:49 dist-test-slave-6zbq krb5kdc[2856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208149, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:55:49.552727 3237 init.cc:377] Logged in from keytab as kudu/127.8.24.2@KRBTEST.COM (short username kudu)
I20260808 16:55:49.553124 3237 webserver.cc:460] Webserver started at http://127.8.24.2:37533/ using document root <none> and password file <none>
I20260808 16:55:49.553334 3237 fs_manager.cc:359] Metadata directory not provided
I20260808 16:55:49.553377 3237 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:55:49.553485 3237 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:55:49.554422 3237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786207919384338-8288-0/minicluster-data/ts-1/data/instance:
uuid: "b0c68a22e5894fe392c273aa2990202d"
format_stamp: "Formatted at 2026-08-08 16:55:49 on dist-test-slave-6zbq"
server_key: "5d5528bcd486807d7293e55e9e4a6c7a"
server_key_iv: "55b730167ca48a437c0350f613fbb3b2"
server_key_version: "encryptionkey@0"
I20260808 16:55:49.554718 3237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786207919384338-8288-0/minicluster-data/ts-1/wal/instance:
uuid: "b0c68a22e5894fe392c273aa2990202d"
format_stamp: "Formatted at 2026-08-08 16:55:49 on dist-test-slave-6zbq"
server_key: "5d5528bcd486807d7293e55e9e4a6c7a"
server_key_iv: "55b730167ca48a437c0350f613fbb3b2"
server_key_version: "encryptionkey@0"
I20260808 16:55:49.555845 3237 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 16:55:49.556437 3256 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:55:49.556578 3237 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 16:55:49.556651 3237 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786207919384338-8288-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786207919384338-8288-0/minicluster-data/ts-1/wal
uuid: "b0c68a22e5894fe392c273aa2990202d"
format_stamp: "Formatted at 2026-08-08 16:55:49 on dist-test-slave-6zbq"
server_key: "5d5528bcd486807d7293e55e9e4a6c7a"
server_key_iv: "55b730167ca48a437c0350f613fbb3b2"
server_key_version: "encryptionkey@0"
I20260808 16:55:49.556712 3237 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786207919384338-8288-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786207919384338-8288-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786207919384338-8288-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:55:49.576467 3237 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:55:49.578003 3237 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:55:49.578119 3237 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:55:49.578310 3237 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:55:49.578599 3237 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:55:49.578632 3237 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:55:49.578651 3237 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:55:49.578662 3237 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:55:49.583847 3237 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.2:45541
I20260808 16:55:49.583906 3437 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.2:45541 every 8 connection(s)
I20260808 16:55:49.584160 3237 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786207919384338-8288-0/minicluster-data/ts-1/data/info.pb
I20260808 16:55:49.588616 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 3237
I20260808 16:55:49.588708 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786207919384338-8288-0/minicluster-data/ts-1/wal/instance
I20260808 16:55:49.588892 8288 external_mini_cluster.cc:1428] Setting key 777f0296feacaa5758b9cf74b4604650
Aug 08 16:55:49 dist-test-slave-6zbq krb5kdc[2856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208149, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.2@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260808 16:55:49.592454 3015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:49.584777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:37475 (local address 127.8.24.62:38177)
0808 16:55:49.584824 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 16:55:49.584826 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:55:49.585017 (+ 191us) server_negotiation.cc:407] Connection header received
0808 16:55:49.585463 (+ 446us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:55:49.585464 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:55:49.585474 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:55:49.585512 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:55:49.586312 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:49.586795 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:49.587696 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:49.587868 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:49.587909 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:49.589922 (+ 2013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:55:49.589924 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:55:49.589924 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:55:49.589932 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:55:49.591235 (+ 1303us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:55:49.591650 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:55:49.591650 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:55:49.591651 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:55:49.591699 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:55:49.591924 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:55:49.591924 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:55:49.591925 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:55:49.592015 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:55:49.592058 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0808 16:55:49.592335 (+ 277us) server_negotiation.cc:299] Negotiation successful
0808 16:55:49.592394 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 16:55:49.592644 3440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:49.584749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:38177 (local address 127.8.24.2:37475)
0808 16:55:49.584914 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:49.584947 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 16:55:49.585377 (+ 430us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:55:49.585561 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:55:49.585564 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:55:49.586070 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:55:49.586251 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:49.586255 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:49.586854 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:49.586854 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:49.587637 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:49.587639 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:49.587919 (+ 280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:49.587919 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:49.588012 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:49.588430 (+ 418us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:55:49.588438 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:55:49.589825 (+ 1387us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:55:49.591318 (+ 1493us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:55:49.591324 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:55:49.591327 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:55:49.591585 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:55:49.591748 (+ 163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:55:49.591748 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:55:49.591749 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:55:49.591870 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:55:49.592063 (+ 193us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:55:49.592067 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:55:49.592210 (+ 143us) client_negotiation.cc:769] Sending connection context
0808 16:55:49.592316 (+ 106us) client_negotiation.cc:240] Negotiation successful
0808 16:55:49.592461 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":97,"threads_started":1}
I20260808 16:55:49.593088 3438 heartbeater.cc:347] Connected to a master server at 127.8.24.62:38177
I20260808 16:55:49.593178 3438 heartbeater.cc:464] Registering TS with master...
I20260808 16:55:49.593340 3438 heartbeater.cc:511] Master 127.8.24.62:38177 requested a full tablet report, sending...
I20260808 16:55:49.593685 2938 ts_manager.cc:194] Registered new tserver with Master: b0c68a22e5894fe392c273aa2990202d (127.8.24.2:45541)
I20260808 16:55:49.594023 2938 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.2@KRBTEST.COM'} at 127.8.24.2:37475
WARNING: no policy specified for kudu/127.8.24.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.3.keytab.
Entry for principal kudu/127.8.24.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.3.keytab.
I20260808 16:55:49.610812 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786207919384338-8288-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786207919384338-8288-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786207919384338-8288-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786207919384338-8288-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.3:0
--local_ip_for_outbound_sockets=127.8.24.3
--webserver_interface=127.8.24.3
--webserver_port=0
--tserver_master_addrs=127.8.24.62:38177
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.3.keytab
--principal=kudu/127.8.24.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:39795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:55:49.688665 3445 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:55:49.688846 3445 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:55:49.688876 3445 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:55:49.689967 3445 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:55:49.690006 3445 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:55:49.690040 3445 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.3
I20260808 16:55:49.691425 3445 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:39795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786207919384338-8288-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786207919384338-8288-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.3.keytab
--principal=kudu/127.8.24.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786207919384338-8288-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.24.3
--webserver_port=0
--tserver_master_addrs=127.8.24.62:38177
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3445
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.3
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786207919384338-8288-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:55:49.691649 3445 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:55:49.691841 3445 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:55:49.696336 3458 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:55:49.696373 3462 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:55:49.696380 3456 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:55:49.696516 3445 server_base.cc:1034] running on GCE node
I20260808 16:55:49.696678 3445 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:55:49.696856 3445 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:55:49.697988 3445 hybrid_clock.cc:648] HybridClock initialized: now 1786208149697980 us; error 27 us; skew 500 ppm
Aug 08 16:55:49 dist-test-slave-6zbq krb5kdc[2856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208149, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:55:49.700251 3445 init.cc:377] Logged in from keytab as kudu/127.8.24.3@KRBTEST.COM (short username kudu)
I20260808 16:55:49.700613 3445 webserver.cc:460] Webserver started at http://127.8.24.3:45689/ using document root <none> and password file <none>
I20260808 16:55:49.700773 3445 fs_manager.cc:359] Metadata directory not provided
I20260808 16:55:49.700819 3445 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:55:49.700906 3445 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:55:49.701818 3445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786207919384338-8288-0/minicluster-data/ts-2/data/instance:
uuid: "ebcec783e2554d7aaee117119602980c"
format_stamp: "Formatted at 2026-08-08 16:55:49 on dist-test-slave-6zbq"
server_key: "c6bbd8cb96946a7f93190125d26815a9"
server_key_iv: "9e0c6d688bef7b3b685cd7e1c8df6c54"
server_key_version: "encryptionkey@0"
I20260808 16:55:49.702093 3445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786207919384338-8288-0/minicluster-data/ts-2/wal/instance:
uuid: "ebcec783e2554d7aaee117119602980c"
format_stamp: "Formatted at 2026-08-08 16:55:49 on dist-test-slave-6zbq"
server_key: "c6bbd8cb96946a7f93190125d26815a9"
server_key_iv: "9e0c6d688bef7b3b685cd7e1c8df6c54"
server_key_version: "encryptionkey@0"
I20260808 16:55:49.703251 3445 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 16:55:49.703845 3472 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:55:49.703984 3445 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 16:55:49.704061 3445 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786207919384338-8288-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786207919384338-8288-0/minicluster-data/ts-2/wal
uuid: "ebcec783e2554d7aaee117119602980c"
format_stamp: "Formatted at 2026-08-08 16:55:49 on dist-test-slave-6zbq"
server_key: "c6bbd8cb96946a7f93190125d26815a9"
server_key_iv: "9e0c6d688bef7b3b685cd7e1c8df6c54"
server_key_version: "encryptionkey@0"
I20260808 16:55:49.704113 3445 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786207919384338-8288-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786207919384338-8288-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786207919384338-8288-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:55:49.715696 3445 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:55:49.717238 3445 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:55:49.717334 3445 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:55:49.717492 3445 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:55:49.717716 3445 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:55:49.717744 3445 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:55:49.717767 3445 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:55:49.717784 3445 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:55:49.723138 3445 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.3:36083
I20260808 16:55:49.723199 3666 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.3:36083 every 8 connection(s)
I20260808 16:55:49.723434 3445 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786207919384338-8288-0/minicluster-data/ts-2/data/info.pb
I20260808 16:55:49.724614 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 3445
I20260808 16:55:49.724705 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786207919384338-8288-0/minicluster-data/ts-2/wal/instance
I20260808 16:55:49.724917 8288 external_mini_cluster.cc:1428] Setting key ec91f2e1bcbe4055b9332b0ff8423f83
Aug 08 16:55:49 dist-test-slave-6zbq krb5kdc[2856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208149, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.3@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260808 16:55:49.731534 3015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:49.724068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:33643 (local address 127.8.24.62:38177)
0808 16:55:49.724129 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 16:55:49.724131 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:55:49.724324 (+ 193us) server_negotiation.cc:407] Connection header received
0808 16:55:49.724754 (+ 430us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:55:49.724756 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:55:49.724769 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:55:49.724844 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:55:49.725845 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:49.726327 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:49.727267 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:49.727450 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:49.727494 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:49.729258 (+ 1764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:55:49.729260 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:55:49.729261 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:55:49.729267 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:55:49.730537 (+ 1270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:55:49.730854 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:55:49.730855 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:55:49.730855 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:55:49.730904 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:55:49.731082 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:55:49.731083 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:55:49.731083 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:55:49.731176 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:55:49.731218 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0808 16:55:49.731408 (+ 190us) server_negotiation.cc:299] Negotiation successful
0808 16:55:49.731470 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 16:55:49.731663 3669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:49.724040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:38177 (local address 127.8.24.3:33643)
0808 16:55:49.724223 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:49.724254 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0808 16:55:49.724656 (+ 402us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:55:49.724899 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:55:49.724902 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:55:49.725551 (+ 649us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:55:49.725765 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:49.725771 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:49.726434 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:49.726435 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:49.727203 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:49.727206 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:49.727501 (+ 295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:49.727502 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:49.727587 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:49.727944 (+ 357us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:55:49.727952 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:55:49.729171 (+ 1219us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:55:49.730595 (+ 1424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:55:49.730600 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:55:49.730601 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:55:49.730795 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:55:49.730948 (+ 153us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:55:49.730949 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:55:49.730949 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:55:49.731037 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:55:49.731216 (+ 179us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:55:49.731218 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:55:49.731319 (+ 101us) client_negotiation.cc:769] Sending connection context
0808 16:55:49.731391 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 16:55:49.731501 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":98,"threads_started":1}
I20260808 16:55:49.732079 3667 heartbeater.cc:347] Connected to a master server at 127.8.24.62:38177
I20260808 16:55:49.732162 3667 heartbeater.cc:464] Registering TS with master...
I20260808 16:55:49.732328 3667 heartbeater.cc:511] Master 127.8.24.62:38177 requested a full tablet report, sending...
I20260808 16:55:49.732666 2938 ts_manager.cc:194] Registered new tserver with Master: ebcec783e2554d7aaee117119602980c (127.8.24.3:36083)
I20260808 16:55:49.733016 2938 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.3@KRBTEST.COM'} at 127.8.24.3:33643
I20260808 16:55:49.736400 8288 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 16:55:49.742820 3015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:49.737630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49670 (local address 127.8.24.62:38177)
0808 16:55:49.737675 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 16:55:49.737677 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:55:49.737772 (+ 95us) server_negotiation.cc:407] Connection header received
0808 16:55:49.737834 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:55:49.737835 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:55:49.737844 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:55:49.737888 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:55:49.738421 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:49.738883 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:49.739576 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:49.739770 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:49.739828 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:49.740792 (+ 964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:55:49.740794 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:55:49.740794 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:55:49.740800 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:55:49.741756 (+ 956us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:55:49.742050 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:55:49.742050 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:55:49.742051 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:55:49.742103 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:55:49.742279 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:55:49.742280 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:55:49.742280 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:55:49.742467 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:55:49.742554 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 16:55:49.742658 (+ 104us) server_negotiation.cc:299] Negotiation successful
0808 16:55:49.742730 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 16:55:49.744241 2938 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49670:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 16:55:49.745837 2938 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 16:55:49.751516 3683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:49.747914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:46153 (local address 127.0.0.1:42398)
0808 16:55:49.748103 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:49.748112 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 16:55:49.748146 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:55:49.748449 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:55:49.748449 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:55:49.748452 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:55:49.748493 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:49.748494 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:49.749403 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:49.749403 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:49.750584 (+ 1181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:49.750586 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:49.751246 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:49.751247 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:49.751320 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:49.751333 (+ 13us) client_negotiation.cc:769] Sending connection context
0808 16:55:49.751394 (+ 61us) client_negotiation.cc:240] Negotiation successful
0808 16:55:49.751443 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":70,"threads_started":1}
I20260808 16:55:49.751533 3686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:49.748156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42398 (local address 127.8.24.1:46153)
0808 16:55:49.748304 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 16:55:49.748309 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 16:55:49.748318 (+ 9us) server_negotiation.cc:407] Connection header received
0808 16:55:49.748353 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:55:49.748355 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:55:49.748369 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:55:49.748512 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:55:49.748587 (+ 75us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:49.749307 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:49.750675 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:49.751166 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:49.751229 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:49.751286 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 16:55:49.751414 (+ 128us) server_negotiation.cc:299] Negotiation successful
0808 16:55:49.751457 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":74,"threads_started":1}
I20260808 16:55:49.751674 3678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:49.747716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.2:45541 (local address 127.0.0.1:53298)
0808 16:55:49.747819 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:49.747853 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0808 16:55:49.747917 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:55:49.748341 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:55:49.748342 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:55:49.748345 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:55:49.748390 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:49.748392 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:49.749693 (+ 1301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:49.749694 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:49.750495 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:49.750498 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:49.751458 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:49.751458 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:49.751521 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:49.751525 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 16:55:49.751570 (+ 45us) client_negotiation.cc:240] Negotiation successful
0808 16:55:49.751606 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77,"thread_start_us":53,"threads_started":1}
I20260808 16:55:49.751783 3680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:49.747716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.3:36083 (local address 127.0.0.1:39450)
0808 16:55:49.747993 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:49.748002 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 16:55:49.748039 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:55:49.748218 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:55:49.748221 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:55:49.748223 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:55:49.748284 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:49.748286 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:49.749616 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:49.749616 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:49.750687 (+ 1071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:49.750689 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:49.751535 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:49.751536 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:49.751588 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:49.751591 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 16:55:49.751630 (+ 39us) client_negotiation.cc:240] Negotiation successful
0808 16:55:49.751662 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"mutex_wait_us":2,"thread_start_us":53,"threads_started":1}
I20260808 16:55:49.751824 3682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:49.747865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39450 (local address 127.8.24.3:36083)
0808 16:55:49.748035 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 16:55:49.748040 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 16:55:49.748051 (+ 11us) server_negotiation.cc:407] Connection header received
0808 16:55:49.748087 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:55:49.748089 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:55:49.748103 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:55:49.748353 (+ 250us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:55:49.748407 (+ 54us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:49.749513 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:49.750759 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:49.751404 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:49.751460 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:49.751506 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0808 16:55:49.751715 (+ 209us) server_negotiation.cc:299] Negotiation successful
0808 16:55:49.751753 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":66,"threads_started":1}
I20260808 16:55:49.752110 3681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:49.747824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53298 (local address 127.8.24.2:45541)
0808 16:55:49.748189 (+ 365us) server_negotiation.cc:206] Beginning negotiation
0808 16:55:49.748193 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 16:55:49.748202 (+ 9us) server_negotiation.cc:407] Connection header received
0808 16:55:49.748238 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:55:49.748241 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:55:49.748255 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:55:49.748345 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:55:49.748570 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:49.749617 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:49.750578 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:49.751299 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:49.751367 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:49.751420 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 16:55:49.751995 (+ 575us) server_negotiation.cc:299] Negotiation successful
0808 16:55:49.752032 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":78,"threads_started":1}
I20260808 16:55:49.752606 3118 tablet_service.cc:1467] Processing CreateTablet for tablet c5f420b53e95438791f012ec21293ad0 (DEFAULT_TABLE table=test-table [id=a23d2d52693149ef990a7648f938b9ef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:55:49.752601 3552 tablet_service.cc:1467] Processing CreateTablet for tablet c5f420b53e95438791f012ec21293ad0 (DEFAULT_TABLE table=test-table [id=a23d2d52693149ef990a7648f938b9ef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:55:49.752686 3337 tablet_service.cc:1467] Processing CreateTablet for tablet c5f420b53e95438791f012ec21293ad0 (DEFAULT_TABLE table=test-table [id=a23d2d52693149ef990a7648f938b9ef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:55:49.752915 3118 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c5f420b53e95438791f012ec21293ad0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:55:49.752934 3552 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c5f420b53e95438791f012ec21293ad0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:55:49.752964 3337 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c5f420b53e95438791f012ec21293ad0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:55:49.755434 3688 tablet_bootstrap.cc:492] T c5f420b53e95438791f012ec21293ad0 P 05caaa0ef0484dc8bae1c3dede53c76a: Bootstrap starting.
I20260808 16:55:49.756044 3692 tablet_bootstrap.cc:492] T c5f420b53e95438791f012ec21293ad0 P b0c68a22e5894fe392c273aa2990202d: Bootstrap starting.
I20260808 16:55:49.756134 3688 tablet_bootstrap.cc:654] T c5f420b53e95438791f012ec21293ad0 P 05caaa0ef0484dc8bae1c3dede53c76a: Neither blocks nor log segments found. Creating new log.
I20260808 16:55:49.756253 3693 tablet_bootstrap.cc:492] T c5f420b53e95438791f012ec21293ad0 P ebcec783e2554d7aaee117119602980c: Bootstrap starting.
I20260808 16:55:49.756722 3692 tablet_bootstrap.cc:654] T c5f420b53e95438791f012ec21293ad0 P b0c68a22e5894fe392c273aa2990202d: Neither blocks nor log segments found. Creating new log.
I20260808 16:55:49.756726 3688 log.cc:824] T c5f420b53e95438791f012ec21293ad0 P 05caaa0ef0484dc8bae1c3dede53c76a: Log is configured to *not* fsync() on all Append() calls
I20260808 16:55:49.756769 3693 tablet_bootstrap.cc:654] T c5f420b53e95438791f012ec21293ad0 P ebcec783e2554d7aaee117119602980c: Neither blocks nor log segments found. Creating new log.
I20260808 16:55:49.757191 3692 log.cc:824] T c5f420b53e95438791f012ec21293ad0 P b0c68a22e5894fe392c273aa2990202d: Log is configured to *not* fsync() on all Append() calls
I20260808 16:55:49.757303 3693 log.cc:824] T c5f420b53e95438791f012ec21293ad0 P ebcec783e2554d7aaee117119602980c: Log is configured to *not* fsync() on all Append() calls
I20260808 16:55:49.757671 3688 tablet_bootstrap.cc:492] T c5f420b53e95438791f012ec21293ad0 P 05caaa0ef0484dc8bae1c3dede53c76a: No bootstrap required, opened a new log
I20260808 16:55:49.757745 3688 ts_tablet_manager.cc:1397] T c5f420b53e95438791f012ec21293ad0 P 05caaa0ef0484dc8bae1c3dede53c76a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 16:55:49.758047 3693 tablet_bootstrap.cc:492] T c5f420b53e95438791f012ec21293ad0 P ebcec783e2554d7aaee117119602980c: No bootstrap required, opened a new log
I20260808 16:55:49.758101 3693 ts_tablet_manager.cc:1397] T c5f420b53e95438791f012ec21293ad0 P ebcec783e2554d7aaee117119602980c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 16:55:49.758908 3692 tablet_bootstrap.cc:492] T c5f420b53e95438791f012ec21293ad0 P b0c68a22e5894fe392c273aa2990202d: No bootstrap required, opened a new log
I20260808 16:55:49.758980 3692 ts_tablet_manager.cc:1397] T c5f420b53e95438791f012ec21293ad0 P b0c68a22e5894fe392c273aa2990202d: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260808 16:55:49.759282 3688 raft_consensus.cc:348] T c5f420b53e95438791f012ec21293ad0 P 05caaa0ef0484dc8bae1c3dede53c76a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebcec783e2554d7aaee117119602980c" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 36083 } } peers { permanent_uuid: "05caaa0ef0484dc8bae1c3dede53c76a" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 46153 } } peers { permanent_uuid: "b0c68a22e5894fe392c273aa2990202d" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 45541 } }
I20260808 16:55:49.759406 3688 raft_consensus.cc:374] T c5f420b53e95438791f012ec21293ad0 P 05caaa0ef0484dc8bae1c3dede53c76a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:55:49.759426 3688 raft_consensus.cc:729] T c5f420b53e95438791f012ec21293ad0 P 05caaa0ef0484dc8bae1c3dede53c76a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05caaa0ef0484dc8bae1c3dede53c76a, State: Initialized, Role: FOLLOWER
I20260808 16:55:49.759518 3688 consensus_queue.cc:260] T c5f420b53e95438791f012ec21293ad0 P 05caaa0ef0484dc8bae1c3dede53c76a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebcec783e2554d7aaee117119602980c" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 36083 } } peers { permanent_uuid: "05caaa0ef0484dc8bae1c3dede53c76a" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 46153 } } peers { permanent_uuid: "b0c68a22e5894fe392c273aa2990202d" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 45541 } }
I20260808 16:55:49.759759 3688 ts_tablet_manager.cc:1428] T c5f420b53e95438791f012ec21293ad0 P 05caaa0ef0484dc8bae1c3dede53c76a: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 16:55:49.759785 3230 heartbeater.cc:503] Master 127.8.24.62:38177 was elected leader, sending a full tablet report...
I20260808 16:55:49.760151 3693 raft_consensus.cc:348] T c5f420b53e95438791f012ec21293ad0 P ebcec783e2554d7aaee117119602980c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebcec783e2554d7aaee117119602980c" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 36083 } } peers { permanent_uuid: "05caaa0ef0484dc8bae1c3dede53c76a" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 46153 } } peers { permanent_uuid: "b0c68a22e5894fe392c273aa2990202d" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 45541 } }
I20260808 16:55:49.760327 3693 raft_consensus.cc:374] T c5f420b53e95438791f012ec21293ad0 P ebcec783e2554d7aaee117119602980c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:55:49.760406 3693 raft_consensus.cc:729] T c5f420b53e95438791f012ec21293ad0 P ebcec783e2554d7aaee117119602980c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ebcec783e2554d7aaee117119602980c, State: Initialized, Role: FOLLOWER
I20260808 16:55:49.760588 3693 consensus_queue.cc:260] T c5f420b53e95438791f012ec21293ad0 P ebcec783e2554d7aaee117119602980c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebcec783e2554d7aaee117119602980c" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 36083 } } peers { permanent_uuid: "05caaa0ef0484dc8bae1c3dede53c76a" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 46153 } } peers { permanent_uuid: "b0c68a22e5894fe392c273aa2990202d" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 45541 } }
I20260808 16:55:49.760849 3693 ts_tablet_manager.cc:1428] T c5f420b53e95438791f012ec21293ad0 P ebcec783e2554d7aaee117119602980c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 16:55:49.760900 3667 heartbeater.cc:503] Master 127.8.24.62:38177 was elected leader, sending a full tablet report...
I20260808 16:55:49.760878 3692 raft_consensus.cc:348] T c5f420b53e95438791f012ec21293ad0 P b0c68a22e5894fe392c273aa2990202d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebcec783e2554d7aaee117119602980c" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 36083 } } peers { permanent_uuid: "05caaa0ef0484dc8bae1c3dede53c76a" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 46153 } } peers { permanent_uuid: "b0c68a22e5894fe392c273aa2990202d" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 45541 } }
I20260808 16:55:49.761034 3692 raft_consensus.cc:374] T c5f420b53e95438791f012ec21293ad0 P b0c68a22e5894fe392c273aa2990202d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:55:49.761060 3692 raft_consensus.cc:729] T c5f420b53e95438791f012ec21293ad0 P b0c68a22e5894fe392c273aa2990202d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0c68a22e5894fe392c273aa2990202d, State: Initialized, Role: FOLLOWER
I20260808 16:55:49.761170 3692 consensus_queue.cc:260] T c5f420b53e95438791f012ec21293ad0 P b0c68a22e5894fe392c273aa2990202d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebcec783e2554d7aaee117119602980c" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 36083 } } peers { permanent_uuid: "05caaa0ef0484dc8bae1c3dede53c76a" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 46153 } } peers { permanent_uuid: "b0c68a22e5894fe392c273aa2990202d" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 45541 } }
I20260808 16:55:49.761431 3692 ts_tablet_manager.cc:1428] T c5f420b53e95438791f012ec21293ad0 P b0c68a22e5894fe392c273aa2990202d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 16:55:49.761541 3438 heartbeater.cc:503] Master 127.8.24.62:38177 was elected leader, sending a full tablet report...
W20260808 16:55:49.834651 3439 tablet.cc:2369] T c5f420b53e95438791f012ec21293ad0 P b0c68a22e5894fe392c273aa2990202d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 16:55:49.899989 3700 raft_consensus.cc:482] T c5f420b53e95438791f012ec21293ad0 P b0c68a22e5894fe392c273aa2990202d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 16:55:49.900099 3700 raft_consensus.cc:504] T c5f420b53e95438791f012ec21293ad0 P b0c68a22e5894fe392c273aa2990202d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebcec783e2554d7aaee117119602980c" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 36083 } } peers { permanent_uuid: "05caaa0ef0484dc8bae1c3dede53c76a" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 46153 } } peers { permanent_uuid: "b0c68a22e5894fe392c273aa2990202d" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 45541 } }
I20260808 16:55:49.900396 3700 leader_election.cc:288] T c5f420b53e95438791f012ec21293ad0 P b0c68a22e5894fe392c273aa2990202d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ebcec783e2554d7aaee117119602980c (127.8.24.3:36083), 05caaa0ef0484dc8bae1c3dede53c76a (127.8.24.1:46153)
I20260808 16:55:49.903044 3682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:49.900641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:35675 (local address 127.8.24.3:36083)
0808 16:55:49.900687 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 16:55:49.900689 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:55:49.900697 (+ 8us) server_negotiation.cc:407] Connection header received
0808 16:55:49.900723 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:55:49.900724 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:55:49.900735 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:55:49.900783 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:55:49.900970 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:49.901444 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:49.902331 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:49.902737 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:49.902801 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:49.902812 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 16:55:49.902945 (+ 133us) server_negotiation.cc:299] Negotiation successful
0808 16:55:49.902984 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 16:55:49.903045 3440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:49.900548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.3:36083 (local address 127.8.24.2:35675)
0808 16:55:49.900601 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:49.900614 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 16:55:49.900666 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:55:49.900851 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:55:49.900851 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:55:49.900853 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:55:49.900912 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:49.900914 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:49.901514 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:49.901515 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:49.902247 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:49.902249 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:49.902818 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:49.902819 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:49.902884 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:49.902887 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 16:55:49.902933 (+ 46us) client_negotiation.cc:240] Negotiation successful
0808 16:55:49.902971 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260808 16:55:49.903182 3686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:49.900743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:58213 (local address 127.8.24.1:46153)
0808 16:55:49.900818 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0808 16:55:49.900820 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:55:49.900877 (+ 57us) server_negotiation.cc:407] Connection header received
0808 16:55:49.900947 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:55:49.900948 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:55:49.900960 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:55:49.900999 (+ 39us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:55:49.901158 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:49.901616 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:49.902514 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:49.902882 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:49.902942 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:49.902952 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 16:55:49.903084 (+ 132us) server_negotiation.cc:299] Negotiation successful
0808 16:55:49.903120 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 16:55:49.903198 3701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:49.900651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:46153 (local address 127.8.24.2:58213)
0808 16:55:49.900806 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:49.900830 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0808 16:55:49.900885 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:55:49.901009 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:55:49.901010 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:55:49.901012 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:55:49.901088 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:49.901090 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:49.901689 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:49.901690 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:49.902435 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:49.902437 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:49.902955 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:49.902956 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:49.903027 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:49.903030 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 16:55:49.903074 (+ 44us) client_negotiation.cc:240] Negotiation successful
0808 16:55:49.903109 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":92,"threads_started":1}
I20260808 16:55:49.903380 3591 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c5f420b53e95438791f012ec21293ad0" candidate_uuid: "b0c68a22e5894fe392c273aa2990202d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ebcec783e2554d7aaee117119602980c" is_pre_election: true
I20260808 16:55:49.903518 3591 raft_consensus.cc:2454] T c5f420b53e95438791f012ec21293ad0 P ebcec783e2554d7aaee117119602980c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b0c68a22e5894fe392c273aa2990202d in term 0.
I20260808 16:55:49.903755 3261 leader_election.cc:302] T c5f420b53e95438791f012ec21293ad0 P b0c68a22e5894fe392c273aa2990202d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b0c68a22e5894fe392c273aa2990202d, ebcec783e2554d7aaee117119602980c; no voters:
I20260808 16:55:49.903856 3146 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c5f420b53e95438791f012ec21293ad0" candidate_uuid: "b0c68a22e5894fe392c273aa2990202d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05caaa0ef0484dc8bae1c3dede53c76a" is_pre_election: true
I20260808 16:55:49.903908 3700 raft_consensus.cc:2781] T c5f420b53e95438791f012ec21293ad0 P b0c68a22e5894fe392c273aa2990202d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 16:55:49.903946 3700 raft_consensus.cc:482] T c5f420b53e95438791f012ec21293ad0 P b0c68a22e5894fe392c273aa2990202d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 16:55:49.903966 3700 raft_consensus.cc:3037] T c5f420b53e95438791f012ec21293ad0 P b0c68a22e5894fe392c273aa2990202d [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:55:49.903996 3146 raft_consensus.cc:2454] T c5f420b53e95438791f012ec21293ad0 P 05caaa0ef0484dc8bae1c3dede53c76a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b0c68a22e5894fe392c273aa2990202d in term 0.
I20260808 16:55:49.904677 3700 raft_consensus.cc:504] T c5f420b53e95438791f012ec21293ad0 P b0c68a22e5894fe392c273aa2990202d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebcec783e2554d7aaee117119602980c" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 36083 } } peers { permanent_uuid: "05caaa0ef0484dc8bae1c3dede53c76a" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 46153 } } peers { permanent_uuid: "b0c68a22e5894fe392c273aa2990202d" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 45541 } }
I20260808 16:55:49.904794 3700 leader_election.cc:288] T c5f420b53e95438791f012ec21293ad0 P b0c68a22e5894fe392c273aa2990202d [CANDIDATE]: Term 1 election: Requested vote from peers ebcec783e2554d7aaee117119602980c (127.8.24.3:36083), 05caaa0ef0484dc8bae1c3dede53c76a (127.8.24.1:46153)
I20260808 16:55:49.904932 3591 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c5f420b53e95438791f012ec21293ad0" candidate_uuid: "b0c68a22e5894fe392c273aa2990202d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ebcec783e2554d7aaee117119602980c"
I20260808 16:55:49.904951 3146 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c5f420b53e95438791f012ec21293ad0" candidate_uuid: "b0c68a22e5894fe392c273aa2990202d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05caaa0ef0484dc8bae1c3dede53c76a"
I20260808 16:55:49.905004 3591 raft_consensus.cc:3037] T c5f420b53e95438791f012ec21293ad0 P ebcec783e2554d7aaee117119602980c [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:55:49.905022 3146 raft_consensus.cc:3037] T c5f420b53e95438791f012ec21293ad0 P 05caaa0ef0484dc8bae1c3dede53c76a [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:55:49.905678 3146 raft_consensus.cc:2454] T c5f420b53e95438791f012ec21293ad0 P 05caaa0ef0484dc8bae1c3dede53c76a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0c68a22e5894fe392c273aa2990202d in term 1.
I20260808 16:55:49.905694 3591 raft_consensus.cc:2454] T c5f420b53e95438791f012ec21293ad0 P ebcec783e2554d7aaee117119602980c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0c68a22e5894fe392c273aa2990202d in term 1.
I20260808 16:55:49.905850 3261 leader_election.cc:302] T c5f420b53e95438791f012ec21293ad0 P b0c68a22e5894fe392c273aa2990202d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05caaa0ef0484dc8bae1c3dede53c76a, b0c68a22e5894fe392c273aa2990202d; no voters:
I20260808 16:55:49.905953 3700 raft_consensus.cc:2781] T c5f420b53e95438791f012ec21293ad0 P b0c68a22e5894fe392c273aa2990202d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:55:49.906088 3700 raft_consensus.cc:686] T c5f420b53e95438791f012ec21293ad0 P b0c68a22e5894fe392c273aa2990202d [term 1 LEADER]: Becoming Leader. State: Replica: b0c68a22e5894fe392c273aa2990202d, State: Running, Role: LEADER
I20260808 16:55:49.906174 3700 consensus_queue.cc:237] T c5f420b53e95438791f012ec21293ad0 P b0c68a22e5894fe392c273aa2990202d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebcec783e2554d7aaee117119602980c" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 36083 } } peers { permanent_uuid: "05caaa0ef0484dc8bae1c3dede53c76a" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 46153 } } peers { permanent_uuid: "b0c68a22e5894fe392c273aa2990202d" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 45541 } }
I20260808 16:55:49.906978 2938 catalog_manager.cc:5495] T c5f420b53e95438791f012ec21293ad0 P b0c68a22e5894fe392c273aa2990202d reported cstate change: term changed from 0 to 1, leader changed from <none> to b0c68a22e5894fe392c273aa2990202d (127.8.24.2). New cstate: current_term: 1 leader_uuid: "b0c68a22e5894fe392c273aa2990202d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebcec783e2554d7aaee117119602980c" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 36083 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05caaa0ef0484dc8bae1c3dede53c76a" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 46153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b0c68a22e5894fe392c273aa2990202d" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 45541 } health_report { overall_health: HEALTHY } } }
I20260808 16:55:49.931960 3681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:49.929823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53304 (local address 127.8.24.2:45541)
0808 16:55:49.929873 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 16:55:49.929875 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:55:49.929883 (+ 8us) server_negotiation.cc:407] Connection header received
0808 16:55:49.929933 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:55:49.929933 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:55:49.929951 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:55:49.930013 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 16:55:49.930125 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:49.930577 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:49.931277 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:49.931474 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:49.931526 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:49.931664 (+ 138us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 16:55:49.931753 (+ 89us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 16:55:49.931820 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 16:55:49.931867 (+ 47us) server_negotiation.cc:299] Negotiation successful
0808 16:55:49.931883 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 16:55:49.933384 3591 raft_consensus.cc:1261] T c5f420b53e95438791f012ec21293ad0 P ebcec783e2554d7aaee117119602980c [term 1 FOLLOWER]: Refusing update from remote peer b0c68a22e5894fe392c273aa2990202d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:55:49.933410 3146 raft_consensus.cc:1261] T c5f420b53e95438791f012ec21293ad0 P 05caaa0ef0484dc8bae1c3dede53c76a [term 1 FOLLOWER]: Refusing update from remote peer b0c68a22e5894fe392c273aa2990202d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:55:49.933702 3702 consensus_queue.cc:1035] T c5f420b53e95438791f012ec21293ad0 P b0c68a22e5894fe392c273aa2990202d [LEADER]: Connected to new peer: Peer: permanent_uuid: "ebcec783e2554d7aaee117119602980c" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 36083 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:55:49.933815 3700 consensus_queue.cc:1035] T c5f420b53e95438791f012ec21293ad0 P b0c68a22e5894fe392c273aa2990202d [LEADER]: Connected to new peer: Peer: permanent_uuid: "05caaa0ef0484dc8bae1c3dede53c76a" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 46153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 16:55:49.934583 3231 tablet.cc:2369] T c5f420b53e95438791f012ec21293ad0 P 05caaa0ef0484dc8bae1c3dede53c76a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 16:55:49.935745 3708 mvcc.cc:205] Tried to move back new op lower bound from 7316308582124974080 to 7316308582016364544. Current Snapshot: MvccSnapshot[applied={T|T < 7316308582124974080}]
I20260808 16:55:49.936128 3710 mvcc.cc:205] Tried to move back new op lower bound from 7316308582124974080 to 7316308582016364544. Current Snapshot: MvccSnapshot[applied={T|T < 7316308582124974080 or (T in {7316308582124974080})}]
I20260808 16:55:49.936867 3709 mvcc.cc:205] Tried to move back new op lower bound from 7316308582124974080 to 7316308582016364544. Current Snapshot: MvccSnapshot[applied={T|T < 7316308582124974080 or (T in {7316308582124974080})}]
I20260808 16:55:49.937938 2933 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49670:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 16:55:49.938030 2933 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49670:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 16:55:49.938735 2933 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e8b2cffc97a04532abae36dfc65ce170: Sending DeleteTablet for 3 replicas of tablet c5f420b53e95438791f012ec21293ad0
I20260808 16:55:49.939034 3552 tablet_service.cc:1514] Processing DeleteTablet for tablet c5f420b53e95438791f012ec21293ad0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:55:49 UTC) from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:39450
I20260808 16:55:49.939047 3118 tablet_service.cc:1514] Processing DeleteTablet for tablet c5f420b53e95438791f012ec21293ad0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:55:49 UTC) from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:42398
I20260808 16:55:49.939265 3721 tablet_replica.cc:330] T c5f420b53e95438791f012ec21293ad0 P ebcec783e2554d7aaee117119602980c: stopping tablet replica
I20260808 16:55:49.939301 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 3031
I20260808 16:55:49.939374 3721 raft_consensus.cc:2229] T c5f420b53e95438791f012ec21293ad0 P ebcec783e2554d7aaee117119602980c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 16:55:49.939364 3337 tablet_service.cc:1514] Processing DeleteTablet for tablet c5f420b53e95438791f012ec21293ad0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:55:49 UTC) from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:53298
I20260808 16:55:49.939461 3721 raft_consensus.cc:2258] T c5f420b53e95438791f012ec21293ad0 P ebcec783e2554d7aaee117119602980c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:55:49.945549 3721 ts_tablet_manager.cc:1905] T c5f420b53e95438791f012ec21293ad0 P ebcec783e2554d7aaee117119602980c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:55:49.947182 3721 ts_tablet_manager.cc:1918] T c5f420b53e95438791f012ec21293ad0 P ebcec783e2554d7aaee117119602980c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 16:55:49.947235 3721 log.cc:1196] T c5f420b53e95438791f012ec21293ad0 P ebcec783e2554d7aaee117119602980c: Deleting WAL directory at /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786207919384338-8288-0/minicluster-data/ts-2/wal/wals/c5f420b53e95438791f012ec21293ad0
I20260808 16:55:49.947444 3721 ts_tablet_manager.cc:1939] T c5f420b53e95438791f012ec21293ad0 P ebcec783e2554d7aaee117119602980c: Deleting consensus metadata
I20260808 16:55:49.949141 2913 catalog_manager.cc:4841] TS ebcec783e2554d7aaee117119602980c (127.8.24.3:36083): tablet c5f420b53e95438791f012ec21293ad0 (table test-table [id=a23d2d52693149ef990a7648f938b9ef]) successfully deleted
I20260808 16:55:49.951960 3724 tablet_replica.cc:330] T c5f420b53e95438791f012ec21293ad0 P b0c68a22e5894fe392c273aa2990202d: stopping tablet replica
I20260808 16:55:49.952031 3724 raft_consensus.cc:2229] T c5f420b53e95438791f012ec21293ad0 P b0c68a22e5894fe392c273aa2990202d [term 1 LEADER]: Raft consensus shutting down.
I20260808 16:55:49.952111 3724 raft_consensus.cc:2258] T c5f420b53e95438791f012ec21293ad0 P b0c68a22e5894fe392c273aa2990202d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:55:49.952941 3724 ts_tablet_manager.cc:1905] T c5f420b53e95438791f012ec21293ad0 P b0c68a22e5894fe392c273aa2990202d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:55:49.954382 3724 ts_tablet_manager.cc:1918] T c5f420b53e95438791f012ec21293ad0 P b0c68a22e5894fe392c273aa2990202d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 16:55:49.954456 3724 log.cc:1196] T c5f420b53e95438791f012ec21293ad0 P b0c68a22e5894fe392c273aa2990202d: Deleting WAL directory at /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786207919384338-8288-0/minicluster-data/ts-1/wal/wals/c5f420b53e95438791f012ec21293ad0
I20260808 16:55:49.954770 3724 ts_tablet_manager.cc:1939] T c5f420b53e95438791f012ec21293ad0 P b0c68a22e5894fe392c273aa2990202d: Deleting consensus metadata
I20260808 16:55:49.956113 2912 catalog_manager.cc:4841] TS b0c68a22e5894fe392c273aa2990202d (127.8.24.2:45541): tablet c5f420b53e95438791f012ec21293ad0 (table test-table [id=a23d2d52693149ef990a7648f938b9ef]) successfully deleted
W20260808 16:55:49.960943 2913 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.24.1:46153 (error 108)
I20260808 16:55:49.961208 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 3237
I20260808 16:55:49.963562 3678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:49.961266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:46153 (local address 127.0.0.1:42412)
0808 16:55:49.963483 (+ 2217us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:49.963528 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.1:46153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2195}
W20260808 16:55:49.963725 2913 catalog_manager.cc:4584] TS 05caaa0ef0484dc8bae1c3dede53c76a (127.8.24.1:46153): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c5f420b53e95438791f012ec21293ad0: Network error: Client connection negotiation failed: client connection to 127.8.24.1:46153: connect: Connection refused (error 111)
I20260808 16:55:49.969925 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 3445
I20260808 16:55:49.976373 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 2882
2026-08-08T16:55:49Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2069 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-08T16:55:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T16:55:50Z Disabled control of system clock
I20260808 16:55:50.007687 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786207919384338-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786207919384338-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786207919384338-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:37943
--webserver_interface=127.8.24.62
--webserver_port=0
--builtin_ntp_servers=127.8.24.20:39843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:37943
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 16:55:50.084937 3731 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:55:50.085116 3731 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:55:50.085134 3731 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:55:50.086227 3731 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 16:55:50.086265 3731 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:55:50.086277 3731 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:55:50.086292 3731 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 16:55:50.086316 3731 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 16:55:50.087625 3731 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:39843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786207919384338-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786207919384338-8288-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:37943
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:37943
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.24.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3731
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786207919384338-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:55:50.087837 3731 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:55:50.088081 3731 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 16:55:50.092459 3731 server_base.cc:1034] running on GCE node
W20260808 16:55:50.092443 3748 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:55:50.092449 3741 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:55:50.092432 3744 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:55:50.092823 3731 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:55:50.093063 3731 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:55:50.094197 3731 hybrid_clock.cc:648] HybridClock initialized: now 1786208150094185 us; error 28 us; skew 500 ppm
I20260808 16:55:50.095346 3731 webserver.cc:460] Webserver started at http://127.8.24.62:41879/ using document root <none> and password file <none>
I20260808 16:55:50.095542 3731 fs_manager.cc:359] Metadata directory not provided
I20260808 16:55:50.095587 3731 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:55:50.095710 3731 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:55:50.096699 3731 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786207919384338-8288-0/minicluster-data/master-0/data/instance:
uuid: "64c90fbab96f4a05ba11187842fb1b7d"
format_stamp: "Formatted at 2026-08-08 16:55:50 on dist-test-slave-6zbq"
server_key: "d5c7d907cf8312510c37065637038ea7"
server_key_iv: "01058144aded08f5eaefe6f6b3975659"
server_key_version: "encryptionkey@0"
I20260808 16:55:50.097007 3731 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786207919384338-8288-0/minicluster-data/master-0/wal/instance:
uuid: "64c90fbab96f4a05ba11187842fb1b7d"
format_stamp: "Formatted at 2026-08-08 16:55:50 on dist-test-slave-6zbq"
server_key: "d5c7d907cf8312510c37065637038ea7"
server_key_iv: "01058144aded08f5eaefe6f6b3975659"
server_key_version: "encryptionkey@0"
I20260808 16:55:50.098112 3731 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 16:55:50.098778 3761 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:55:50.098908 3731 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 16:55:50.098982 3731 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786207919384338-8288-0/minicluster-data/master-0/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786207919384338-8288-0/minicluster-data/master-0/wal
uuid: "64c90fbab96f4a05ba11187842fb1b7d"
format_stamp: "Formatted at 2026-08-08 16:55:50 on dist-test-slave-6zbq"
server_key: "d5c7d907cf8312510c37065637038ea7"
server_key_iv: "01058144aded08f5eaefe6f6b3975659"
server_key_version: "encryptionkey@0"
I20260808 16:55:50.099045 3731 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786207919384338-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786207919384338-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786207919384338-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:55:50.120980 3731 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:55:50.121222 3731 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:55:50.121346 3731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:55:50.124712 3731 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.62:37943
I20260808 16:55:50.124758 3846 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.62:37943 every 8 connection(s)
I20260808 16:55:50.125001 3731 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
I20260808 16:55:50.125850 3847 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:55:50.127851 3847 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64c90fbab96f4a05ba11187842fb1b7d: Bootstrap starting.
I20260808 16:55:50.128362 3847 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 64c90fbab96f4a05ba11187842fb1b7d: Neither blocks nor log segments found. Creating new log.
I20260808 16:55:50.128613 3847 log.cc:824] T 00000000000000000000000000000000 P 64c90fbab96f4a05ba11187842fb1b7d: Log is configured to *not* fsync() on all Append() calls
I20260808 16:55:50.129199 3847 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64c90fbab96f4a05ba11187842fb1b7d: No bootstrap required, opened a new log
I20260808 16:55:50.130681 3847 raft_consensus.cc:348] T 00000000000000000000000000000000 P 64c90fbab96f4a05ba11187842fb1b7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64c90fbab96f4a05ba11187842fb1b7d" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 37943 } }
I20260808 16:55:50.130806 3847 raft_consensus.cc:374] T 00000000000000000000000000000000 P 64c90fbab96f4a05ba11187842fb1b7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:55:50.130838 3847 raft_consensus.cc:729] T 00000000000000000000000000000000 P 64c90fbab96f4a05ba11187842fb1b7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64c90fbab96f4a05ba11187842fb1b7d, State: Initialized, Role: FOLLOWER
I20260808 16:55:50.130963 3847 consensus_queue.cc:260] T 00000000000000000000000000000000 P 64c90fbab96f4a05ba11187842fb1b7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64c90fbab96f4a05ba11187842fb1b7d" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 37943 } }
I20260808 16:55:50.131021 3847 raft_consensus.cc:388] T 00000000000000000000000000000000 P 64c90fbab96f4a05ba11187842fb1b7d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 16:55:50.131043 3847 raft_consensus.cc:482] T 00000000000000000000000000000000 P 64c90fbab96f4a05ba11187842fb1b7d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 16:55:50.131085 3847 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 64c90fbab96f4a05ba11187842fb1b7d [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:55:50.131546 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 3731
I20260808 16:55:50.131592 3847 raft_consensus.cc:504] T 00000000000000000000000000000000 P 64c90fbab96f4a05ba11187842fb1b7d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64c90fbab96f4a05ba11187842fb1b7d" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 37943 } }
I20260808 16:55:50.131650 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786207919384338-8288-0/minicluster-data/master-0/wal/instance
I20260808 16:55:50.131703 3847 leader_election.cc:302] T 00000000000000000000000000000000 P 64c90fbab96f4a05ba11187842fb1b7d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 64c90fbab96f4a05ba11187842fb1b7d; no voters:
I20260808 16:55:50.131831 8288 external_mini_cluster.cc:1428] Setting key ffedf32de5a9387b261d2c7c1d29a48d
I20260808 16:55:50.131858 3847 leader_election.cc:288] T 00000000000000000000000000000000 P 64c90fbab96f4a05ba11187842fb1b7d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 16:55:50.131912 3854 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 64c90fbab96f4a05ba11187842fb1b7d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:55:50.132082 3847 sys_catalog.cc:564] T 00000000000000000000000000000000 P 64c90fbab96f4a05ba11187842fb1b7d [sys.catalog]: configured and running, proceeding with master startup.
I20260808 16:55:50.132099 3854 raft_consensus.cc:686] T 00000000000000000000000000000000 P 64c90fbab96f4a05ba11187842fb1b7d [term 1 LEADER]: Becoming Leader. State: Replica: 64c90fbab96f4a05ba11187842fb1b7d, State: Running, Role: LEADER
I20260808 16:55:50.132277 3854 consensus_queue.cc:237] T 00000000000000000000000000000000 P 64c90fbab96f4a05ba11187842fb1b7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64c90fbab96f4a05ba11187842fb1b7d" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 37943 } }
I20260808 16:55:50.132722 3856 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64c90fbab96f4a05ba11187842fb1b7d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "64c90fbab96f4a05ba11187842fb1b7d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64c90fbab96f4a05ba11187842fb1b7d" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 37943 } } }
I20260808 16:55:50.132812 3856 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64c90fbab96f4a05ba11187842fb1b7d [sys.catalog]: This master's current role is: LEADER
I20260808 16:55:50.133029 3857 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64c90fbab96f4a05ba11187842fb1b7d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 64c90fbab96f4a05ba11187842fb1b7d. Latest consensus state: current_term: 1 leader_uuid: "64c90fbab96f4a05ba11187842fb1b7d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64c90fbab96f4a05ba11187842fb1b7d" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 37943 } } }
I20260808 16:55:50.133402 3871 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 16:55:50.133404 3857 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64c90fbab96f4a05ba11187842fb1b7d [sys.catalog]: This master's current role is: LEADER
I20260808 16:55:50.133937 3871 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 16:55:50.137323 3860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:50.132234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53590 (local address 127.8.24.62:37943)
0808 16:55:50.132574 (+ 340us) server_negotiation.cc:206] Beginning negotiation
0808 16:55:50.132582 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 16:55:50.132612 (+ 30us) server_negotiation.cc:407] Connection header received
0808 16:55:50.132726 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:55:50.132732 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:55:50.132797 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:55:50.133208 (+ 411us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:55:50.133319 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:50.134650 (+ 1331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:50.135464 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:50.136064 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:50.136187 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:50.136692 (+ 505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:55:50.136696 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:55:50.136697 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 16:55:50.136881 (+ 184us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:55:50.136921 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 16:55:50.136936 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:55:50.137003 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 16:55:50.137040 (+ 37us) server_negotiation.cc:299] Negotiation successful
0808 16:55:50.137112 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":270,"thread_start_us":80,"threads_started":1}
I20260808 16:55:50.137477 3871 catalog_manager.cc:1347] Generated new cluster ID: 366314097e774675aabd1b62dc773bb8
I20260808 16:55:50.137524 3871 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 16:55:50.161060 3871 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 16:55:50.161496 3871 catalog_manager.cc:1504] Loading token signing keys...
I20260808 16:55:50.164798 3871 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 64c90fbab96f4a05ba11187842fb1b7d: Generated new TSK 0
I20260808 16:55:50.164943 3871 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 16:55:50.167265 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786207919384338-8288-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786207919384338-8288-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786207919384338-8288-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786207919384338-8288-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.1:0
--local_ip_for_outbound_sockets=127.8.24.1
--webserver_interface=127.8.24.1
--webserver_port=0
--tserver_master_addrs=127.8.24.62:37943
--builtin_ntp_servers=127.8.24.20:39843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 16:55:50.267019 3889 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:55:50.267196 3889 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:55:50.267223 3889 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:55:50.268294 3889 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:55:50.268332 3889 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:55:50.268363 3889 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.1
I20260808 16:55:50.269658 3889 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:39843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786207919384338-8288-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786207919384338-8288-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786207919384338-8288-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.24.1
--webserver_port=0
--tserver_master_addrs=127.8.24.62:37943
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3889
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.1
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786207919384338-8288-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:55:50.269874 3889 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:55:50.270117 3889 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:55:50.274569 3898 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:55:50.274580 3897 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:55:50.274574 3900 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:55:50.274899 3889 server_base.cc:1034] running on GCE node
I20260808 16:55:50.275066 3889 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:55:50.275265 3889 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:55:50.276407 3889 hybrid_clock.cc:648] HybridClock initialized: now 1786208150276393 us; error 28 us; skew 500 ppm
I20260808 16:55:50.277490 3889 webserver.cc:460] Webserver started at http://127.8.24.1:33745/ using document root <none> and password file <none>
I20260808 16:55:50.277702 3889 fs_manager.cc:359] Metadata directory not provided
I20260808 16:55:50.277750 3889 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:55:50.277868 3889 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:55:50.278882 3889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786207919384338-8288-0/minicluster-data/ts-0/data/instance:
uuid: "e2384c82f6c24e29952eca100df5819d"
format_stamp: "Formatted at 2026-08-08 16:55:50 on dist-test-slave-6zbq"
server_key: "1fd2f518468a9464216e66eed3d140c0"
server_key_iv: "49975fa9807c1bd00f3bb3b1ec46f466"
server_key_version: "encryptionkey@0"
I20260808 16:55:50.279184 3889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786207919384338-8288-0/minicluster-data/ts-0/wal/instance:
uuid: "e2384c82f6c24e29952eca100df5819d"
format_stamp: "Formatted at 2026-08-08 16:55:50 on dist-test-slave-6zbq"
server_key: "1fd2f518468a9464216e66eed3d140c0"
server_key_iv: "49975fa9807c1bd00f3bb3b1ec46f466"
server_key_version: "encryptionkey@0"
I20260808 16:55:50.280299 3889 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 16:55:50.280905 3910 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:55:50.281046 3889 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 16:55:50.281134 3889 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786207919384338-8288-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786207919384338-8288-0/minicluster-data/ts-0/wal
uuid: "e2384c82f6c24e29952eca100df5819d"
format_stamp: "Formatted at 2026-08-08 16:55:50 on dist-test-slave-6zbq"
server_key: "1fd2f518468a9464216e66eed3d140c0"
server_key_iv: "49975fa9807c1bd00f3bb3b1ec46f466"
server_key_version: "encryptionkey@0"
I20260808 16:55:50.281193 3889 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786207919384338-8288-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786207919384338-8288-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786207919384338-8288-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:55:50.295214 3889 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:55:50.295437 3889 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:55:50.295547 3889 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:55:50.295737 3889 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:55:50.295980 3889 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:55:50.296015 3889 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:55:50.296033 3889 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:55:50.296049 3889 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:55:50.301345 3889 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.1:35259
I20260808 16:55:50.301425 4092 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.1:35259 every 8 connection(s)
I20260808 16:55:50.301659 3889 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786207919384338-8288-0/minicluster-data/ts-0/data/info.pb
I20260808 16:55:50.302747 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 3889
I20260808 16:55:50.302820 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786207919384338-8288-0/minicluster-data/ts-0/wal/instance
I20260808 16:55:50.302965 8288 external_mini_cluster.cc:1428] Setting key 35f8df326ca0be4e0b444cc4f9fb6aea
I20260808 16:55:50.303862 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786207919384338-8288-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786207919384338-8288-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786207919384338-8288-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786207919384338-8288-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.2:0
--local_ip_for_outbound_sockets=127.8.24.2
--webserver_interface=127.8.24.2
--webserver_port=0
--tserver_master_addrs=127.8.24.62:37943
--builtin_ntp_servers=127.8.24.20:39843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 16:55:50.306584 3860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:50.302239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:53543 (local address 127.8.24.62:37943)
0808 16:55:50.302312 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0808 16:55:50.302315 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 16:55:50.302542 (+ 227us) server_negotiation.cc:407] Connection header received
0808 16:55:50.303026 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:55:50.303027 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:55:50.303041 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:55:50.303091 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:55:50.303790 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:50.304319 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:50.305321 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:50.305515 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:50.305570 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:50.306357 (+ 787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:55:50.306358 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:55:50.306358 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 16:55:50.306363 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:55:50.306376 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 16:55:50.306380 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:55:50.306414 (+ 34us) server_negotiation.cc:1035] Waiting for connection context
0808 16:55:50.306509 (+ 95us) server_negotiation.cc:299] Negotiation successful
0808 16:55:50.306530 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 16:55:50.306806 4095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:50.302235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:37943 (local address 127.8.24.1:53543)
0808 16:55:50.302444 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:50.302479 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 16:55:50.302928 (+ 449us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:55:50.303153 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:55:50.303157 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:55:50.303167 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:55:50.303696 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:50.303702 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:50.304405 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:50.304406 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:50.305236 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:50.305239 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:50.305588 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:50.305589 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:50.305732 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:50.306185 (+ 453us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 16:55:50.306185 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:55:50.306220 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 16:55:50.306247 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:55:50.306440 (+ 193us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:55:50.306447 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:55:50.306449 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 16:55:50.306510 (+ 61us) client_negotiation.cc:240] Negotiation successful
0808 16:55:50.306605 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":104,"threads_started":1}
I20260808 16:55:50.307333 4093 heartbeater.cc:347] Connected to a master server at 127.8.24.62:37943
I20260808 16:55:50.307423 4093 heartbeater.cc:464] Registering TS with master...
I20260808 16:55:50.307585 4093 heartbeater.cc:511] Master 127.8.24.62:37943 requested a full tablet report, sending...
I20260808 16:55:50.308105 3792 ts_manager.cc:194] Registered new tserver with Master: e2384c82f6c24e29952eca100df5819d (127.8.24.1:35259)
I20260808 16:55:50.308964 3792 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.24.1:53543
W20260808 16:55:50.382364 4096 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:55:50.382541 4096 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:55:50.382561 4096 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:55:50.383666 4096 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:55:50.383702 4096 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:55:50.383725 4096 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.2
I20260808 16:55:50.385003 4096 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:39843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786207919384338-8288-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786207919384338-8288-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786207919384338-8288-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.24.2
--webserver_port=0
--tserver_master_addrs=127.8.24.62:37943
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4096
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.2
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786207919384338-8288-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:55:50.385170 4096 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:55:50.385386 4096 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:55:50.389796 4102 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:55:50.389883 4096 server_base.cc:1034] running on GCE node
W20260808 16:55:50.389806 4101 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:55:50.389806 4104 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:55:50.390209 4096 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:55:50.390457 4096 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:55:50.391594 4096 hybrid_clock.cc:648] HybridClock initialized: now 1786208150391581 us; error 25 us; skew 500 ppm
I20260808 16:55:50.392670 4096 webserver.cc:460] Webserver started at http://127.8.24.2:34583/ using document root <none> and password file <none>
I20260808 16:55:50.392869 4096 fs_manager.cc:359] Metadata directory not provided
I20260808 16:55:50.392916 4096 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:55:50.393038 4096 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:55:50.393988 4096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786207919384338-8288-0/minicluster-data/ts-1/data/instance:
uuid: "b2b673c9a0b54c71b963c8a414dea61f"
format_stamp: "Formatted at 2026-08-08 16:55:50 on dist-test-slave-6zbq"
server_key: "38edc9fe3354b4bba772401fe155bbd5"
server_key_iv: "674ab46d4c5347e9860e73779176325a"
server_key_version: "encryptionkey@0"
I20260808 16:55:50.394312 4096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786207919384338-8288-0/minicluster-data/ts-1/wal/instance:
uuid: "b2b673c9a0b54c71b963c8a414dea61f"
format_stamp: "Formatted at 2026-08-08 16:55:50 on dist-test-slave-6zbq"
server_key: "38edc9fe3354b4bba772401fe155bbd5"
server_key_iv: "674ab46d4c5347e9860e73779176325a"
server_key_version: "encryptionkey@0"
I20260808 16:55:50.395532 4096 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 16:55:50.396152 4110 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:55:50.396317 4096 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 16:55:50.396396 4096 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786207919384338-8288-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786207919384338-8288-0/minicluster-data/ts-1/wal
uuid: "b2b673c9a0b54c71b963c8a414dea61f"
format_stamp: "Formatted at 2026-08-08 16:55:50 on dist-test-slave-6zbq"
server_key: "38edc9fe3354b4bba772401fe155bbd5"
server_key_iv: "674ab46d4c5347e9860e73779176325a"
server_key_version: "encryptionkey@0"
I20260808 16:55:50.396461 4096 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786207919384338-8288-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786207919384338-8288-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786207919384338-8288-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:55:50.403882 4096 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:55:50.404099 4096 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:55:50.404209 4096 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:55:50.404398 4096 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:55:50.404670 4096 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:55:50.404702 4096 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:55:50.404727 4096 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:55:50.404754 4096 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:55:50.410020 4096 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.2:45313
I20260808 16:55:50.410056 4278 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.2:45313 every 8 connection(s)
I20260808 16:55:50.410332 4096 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786207919384338-8288-0/minicluster-data/ts-1/data/info.pb
I20260808 16:55:50.414963 3860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:50.410995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:40333 (local address 127.8.24.62:37943)
0808 16:55:50.411051 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 16:55:50.411053 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:55:50.411235 (+ 182us) server_negotiation.cc:407] Connection header received
0808 16:55:50.411686 (+ 451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:55:50.411687 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:55:50.411701 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:55:50.411747 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:55:50.412348 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:50.412846 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:50.413749 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:50.413950 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:50.414006 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:50.414709 (+ 703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:55:50.414712 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:55:50.414712 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 16:55:50.414717 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:55:50.414733 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 16:55:50.414737 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:55:50.414783 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0808 16:55:50.414855 (+ 72us) server_negotiation.cc:299] Negotiation successful
0808 16:55:50.414885 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 16:55:50.415119 4281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:50.410976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:37943 (local address 127.8.24.2:40333)
0808 16:55:50.411137 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:50.411166 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0808 16:55:50.411594 (+ 428us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:55:50.411819 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:55:50.411822 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:55:50.411832 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:55:50.412264 (+ 432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:50.412268 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:50.412928 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:50.412929 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:50.413668 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:50.413670 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:50.414022 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:50.414023 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:50.414136 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:50.414584 (+ 448us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 16:55:50.414584 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:55:50.414609 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 16:55:50.414630 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:55:50.414789 (+ 159us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:55:50.414794 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:55:50.414796 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 16:55:50.414854 (+ 58us) client_negotiation.cc:240] Negotiation successful
0808 16:55:50.414946 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":69,"threads_started":1}
I20260808 16:55:50.415508 4279 heartbeater.cc:347] Connected to a master server at 127.8.24.62:37943
I20260808 16:55:50.415589 4279 heartbeater.cc:464] Registering TS with master...
I20260808 16:55:50.415751 4279 heartbeater.cc:511] Master 127.8.24.62:37943 requested a full tablet report, sending...
I20260808 16:55:50.416085 3792 ts_manager.cc:194] Registered new tserver with Master: b2b673c9a0b54c71b963c8a414dea61f (127.8.24.2:45313)
I20260808 16:55:50.416414 3792 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.24.2:40333
I20260808 16:55:50.419456 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 4096
I20260808 16:55:50.419548 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786207919384338-8288-0/minicluster-data/ts-1/wal/instance
I20260808 16:55:50.419711 8288 external_mini_cluster.cc:1428] Setting key 12c7e3d4197e9e918d586a35cb7f91ff
I20260808 16:55:50.420503 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786207919384338-8288-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786207919384338-8288-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786207919384338-8288-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786207919384338-8288-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.3:0
--local_ip_for_outbound_sockets=127.8.24.3
--webserver_interface=127.8.24.3
--webserver_port=0
--tserver_master_addrs=127.8.24.62:37943
--builtin_ntp_servers=127.8.24.20:39843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 16:55:50.498487 4283 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:55:50.498695 4283 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:55:50.498718 4283 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:55:50.499833 4283 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:55:50.499871 4283 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:55:50.499897 4283 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.3
I20260808 16:55:50.501214 4283 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:39843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786207919384338-8288-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786207919384338-8288-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786207919384338-8288-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.24.3
--webserver_port=0
--tserver_master_addrs=127.8.24.62:37943
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4283
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.3
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786207919384338-8288-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:55:50.501410 4283 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:55:50.501678 4283 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:55:50.506060 4289 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:55:50.506217 4283 server_base.cc:1034] running on GCE node
W20260808 16:55:50.506065 4294 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:55:50.506170 4290 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:55:50.506539 4283 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:55:50.506759 4283 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:55:50.507898 4283 hybrid_clock.cc:648] HybridClock initialized: now 1786208150507881 us; error 28 us; skew 500 ppm
I20260808 16:55:50.508970 4283 webserver.cc:460] Webserver started at http://127.8.24.3:33451/ using document root <none> and password file <none>
I20260808 16:55:50.509178 4283 fs_manager.cc:359] Metadata directory not provided
I20260808 16:55:50.509224 4283 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:55:50.509347 4283 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:55:50.510368 4283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786207919384338-8288-0/minicluster-data/ts-2/data/instance:
uuid: "170adff766bf483d9ab90fba2f1e301c"
format_stamp: "Formatted at 2026-08-08 16:55:50 on dist-test-slave-6zbq"
server_key: "e4a892e0abac4605a3f83b244243f7f9"
server_key_iv: "7cc92b79bc83bc3bf60c6498f6927c1a"
server_key_version: "encryptionkey@0"
I20260808 16:55:50.510707 4283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786207919384338-8288-0/minicluster-data/ts-2/wal/instance:
uuid: "170adff766bf483d9ab90fba2f1e301c"
format_stamp: "Formatted at 2026-08-08 16:55:50 on dist-test-slave-6zbq"
server_key: "e4a892e0abac4605a3f83b244243f7f9"
server_key_iv: "7cc92b79bc83bc3bf60c6498f6927c1a"
server_key_version: "encryptionkey@0"
I20260808 16:55:50.511878 4283 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 16:55:50.512501 4303 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:55:50.512655 4283 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 16:55:50.512732 4283 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786207919384338-8288-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786207919384338-8288-0/minicluster-data/ts-2/wal
uuid: "170adff766bf483d9ab90fba2f1e301c"
format_stamp: "Formatted at 2026-08-08 16:55:50 on dist-test-slave-6zbq"
server_key: "e4a892e0abac4605a3f83b244243f7f9"
server_key_iv: "7cc92b79bc83bc3bf60c6498f6927c1a"
server_key_version: "encryptionkey@0"
I20260808 16:55:50.512804 4283 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786207919384338-8288-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786207919384338-8288-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786207919384338-8288-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:55:50.519411 4283 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:55:50.519629 4283 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:55:50.519737 4283 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:55:50.519912 4283 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:55:50.520171 4283 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:55:50.520203 4283 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:55:50.520228 4283 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:55:50.520251 4283 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:55:50.525410 4283 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.3:45039
I20260808 16:55:50.525468 4475 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.3:45039 every 8 connection(s)
I20260808 16:55:50.525707 4283 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786207919384338-8288-0/minicluster-data/ts-2/data/info.pb
I20260808 16:55:50.530148 3860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:50.526280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:51581 (local address 127.8.24.62:37943)
0808 16:55:50.526366 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0808 16:55:50.526369 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 16:55:50.526540 (+ 171us) server_negotiation.cc:407] Connection header received
0808 16:55:50.526961 (+ 421us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:55:50.526963 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:55:50.526976 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:55:50.527026 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:55:50.527621 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:50.528114 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:50.529013 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:50.529220 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:50.529275 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:50.529893 (+ 618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:55:50.529895 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:55:50.529896 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 16:55:50.529900 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:55:50.529917 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 16:55:50.529922 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:55:50.529967 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0808 16:55:50.530040 (+ 73us) server_negotiation.cc:299] Negotiation successful
0808 16:55:50.530071 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 16:55:50.530323 4483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:50.526250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:37943 (local address 127.8.24.3:51581)
0808 16:55:50.526444 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:50.526472 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0808 16:55:50.526873 (+ 401us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:55:50.527087 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:55:50.527091 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:55:50.527100 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:55:50.527539 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:50.527543 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:50.528196 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:50.528197 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:50.528933 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:50.528935 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:50.529299 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:50.529300 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:50.529416 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:50.529771 (+ 355us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 16:55:50.529771 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:55:50.529798 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 16:55:50.529817 (+ 19us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:55:50.529973 (+ 156us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:55:50.529978 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:55:50.529981 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 16:55:50.530038 (+ 57us) client_negotiation.cc:240] Negotiation successful
0808 16:55:50.530128 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":104,"threads_started":1}
I20260808 16:55:50.530764 4477 heartbeater.cc:347] Connected to a master server at 127.8.24.62:37943
I20260808 16:55:50.530843 4477 heartbeater.cc:464] Registering TS with master...
I20260808 16:55:50.530969 4477 heartbeater.cc:511] Master 127.8.24.62:37943 requested a full tablet report, sending...
I20260808 16:55:50.531277 3792 ts_manager.cc:194] Registered new tserver with Master: 170adff766bf483d9ab90fba2f1e301c (127.8.24.3:45039)
I20260808 16:55:50.531610 3792 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.24.3:51581
I20260808 16:55:50.535096 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 4283
I20260808 16:55:50.535171 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786207919384338-8288-0/minicluster-data/ts-2/wal/instance
I20260808 16:55:50.535357 8288 external_mini_cluster.cc:1428] Setting key ce82b8ca81866c2f89d2110e6869ddd3
I20260808 16:55:50.536116 8288 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 16:55:50.539903 3860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:50.537310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53598 (local address 127.8.24.62:37943)
0808 16:55:50.537367 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 16:55:50.537369 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:55:50.537433 (+ 64us) server_negotiation.cc:407] Connection header received
0808 16:55:50.537499 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:55:50.537500 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:55:50.537511 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:55:50.537552 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:55:50.537708 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:50.538144 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:50.538919 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:50.539092 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:50.539143 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:50.539702 (+ 559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:55:50.539703 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:55:50.539703 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 16:55:50.539707 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:55:50.539719 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 16:55:50.539723 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:55:50.539764 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0808 16:55:50.539817 (+ 53us) server_negotiation.cc:299] Negotiation successful
0808 16:55:50.539837 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 16:55:50.541241 3792 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53598:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 16:55:50.542706 3792 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 16:55:50.548355 4492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:50.544481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:35259 (local address 127.0.0.1:49730)
0808 16:55:50.544915 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:50.544925 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 16:55:50.544970 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:55:50.545146 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:55:50.545147 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:55:50.545149 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:55:50.545176 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:50.545177 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:50.546450 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:50.546451 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:50.547204 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:50.547207 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:50.548065 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:50.548066 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:50.548145 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:50.548158 (+ 13us) client_negotiation.cc:769] Sending connection context
0808 16:55:50.548229 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 16:55:50.548278 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":423,"mutex_wait_us":31,"thread_start_us":51,"threads_started":1}
I20260808 16:55:50.548452 4500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:50.544762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46852 (local address 127.8.24.3:45039)
0808 16:55:50.544912 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 16:55:50.544916 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 16:55:50.544926 (+ 10us) server_negotiation.cc:407] Connection header received
0808 16:55:50.544958 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:55:50.544960 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:55:50.544973 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:55:50.545066 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:55:50.545210 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:50.546072 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:50.547452 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:50.548110 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:50.548197 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:50.548273 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 16:55:50.548355 (+ 82us) server_negotiation.cc:299] Negotiation successful
0808 16:55:50.548386 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":86,"threads_started":1}
I20260808 16:55:50.548756 4497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:50.544764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46032 (local address 127.8.24.2:45313)
0808 16:55:50.544915 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0808 16:55:50.544920 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 16:55:50.545148 (+ 228us) server_negotiation.cc:407] Connection header received
0808 16:55:50.545199 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:55:50.545201 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:55:50.545215 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:55:50.545286 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:55:50.545373 (+ 87us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:50.546436 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:50.547632 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:50.548192 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:50.548246 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:50.548326 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 16:55:50.548637 (+ 311us) server_negotiation.cc:299] Negotiation successful
0808 16:55:50.548680 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":74,"threads_started":1}
I20260808 16:55:50.548821 4494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:50.544578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49730 (local address 127.8.24.1:35259)
0808 16:55:50.544958 (+ 380us) server_negotiation.cc:206] Beginning negotiation
0808 16:55:50.544961 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 16:55:50.544971 (+ 10us) server_negotiation.cc:407] Connection header received
0808 16:55:50.545057 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:55:50.545059 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:55:50.545073 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:55:50.545148 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:55:50.545235 (+ 87us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:50.546330 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:50.547286 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:50.547975 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:50.548050 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:50.548134 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 16:55:50.548706 (+ 572us) server_negotiation.cc:299] Negotiation successful
0808 16:55:50.548748 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":92,"threads_started":1}
I20260808 16:55:50.548449 4493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:50.544480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.3:45039 (local address 127.0.0.1:46852)
0808 16:55:50.544627 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:50.544665 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 16:55:50.544725 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:55:50.545075 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:55:50.545077 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:55:50.545079 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:55:50.545135 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:50.545137 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:50.546151 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:50.546152 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:50.547366 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:50.547369 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:50.548222 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:50.548223 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:50.548284 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:50.548288 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 16:55:50.548334 (+ 46us) client_negotiation.cc:240] Negotiation successful
0808 16:55:50.548372 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":83,"threads_started":1}
I20260808 16:55:50.548576 4496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:50.544659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.2:45313 (local address 127.0.0.1:46032)
0808 16:55:50.545024 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:50.545034 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 16:55:50.545091 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:55:50.545290 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:55:50.545290 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:55:50.545292 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:55:50.545318 (+ 26us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:50.545319 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:50.546511 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:50.546512 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:50.547549 (+ 1037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:50.547551 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:50.548362 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:50.548363 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:50.548421 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:50.548426 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 16:55:50.548475 (+ 49us) client_negotiation.cc:240] Negotiation successful
0808 16:55:50.548509 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":347,"mutex_wait_us":1,"thread_start_us":83,"threads_started":1}
I20260808 16:55:50.549580 4374 tablet_service.cc:1467] Processing CreateTablet for tablet ed17abdcf08940c3a3b1cb5a57a775e8 (DEFAULT_TABLE table=test-table [id=596f722e997540649d200d33479c6183]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:55:50.549580 3987 tablet_service.cc:1467] Processing CreateTablet for tablet ed17abdcf08940c3a3b1cb5a57a775e8 (DEFAULT_TABLE table=test-table [id=596f722e997540649d200d33479c6183]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:55:50.549641 4174 tablet_service.cc:1467] Processing CreateTablet for tablet ed17abdcf08940c3a3b1cb5a57a775e8 (DEFAULT_TABLE table=test-table [id=596f722e997540649d200d33479c6183]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:55:50.549911 3987 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ed17abdcf08940c3a3b1cb5a57a775e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:55:50.549911 4374 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ed17abdcf08940c3a3b1cb5a57a775e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:55:50.549932 4174 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ed17abdcf08940c3a3b1cb5a57a775e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:55:50.552381 4501 tablet_bootstrap.cc:492] T ed17abdcf08940c3a3b1cb5a57a775e8 P b2b673c9a0b54c71b963c8a414dea61f: Bootstrap starting.
I20260808 16:55:50.552908 4503 tablet_bootstrap.cc:492] T ed17abdcf08940c3a3b1cb5a57a775e8 P e2384c82f6c24e29952eca100df5819d: Bootstrap starting.
I20260808 16:55:50.552943 4501 tablet_bootstrap.cc:654] T ed17abdcf08940c3a3b1cb5a57a775e8 P b2b673c9a0b54c71b963c8a414dea61f: Neither blocks nor log segments found. Creating new log.
I20260808 16:55:50.553236 4501 log.cc:824] T ed17abdcf08940c3a3b1cb5a57a775e8 P b2b673c9a0b54c71b963c8a414dea61f: Log is configured to *not* fsync() on all Append() calls
I20260808 16:55:50.553592 4503 tablet_bootstrap.cc:654] T ed17abdcf08940c3a3b1cb5a57a775e8 P e2384c82f6c24e29952eca100df5819d: Neither blocks nor log segments found. Creating new log.
I20260808 16:55:50.553876 4503 log.cc:824] T ed17abdcf08940c3a3b1cb5a57a775e8 P e2384c82f6c24e29952eca100df5819d: Log is configured to *not* fsync() on all Append() calls
I20260808 16:55:50.553966 4501 tablet_bootstrap.cc:492] T ed17abdcf08940c3a3b1cb5a57a775e8 P b2b673c9a0b54c71b963c8a414dea61f: No bootstrap required, opened a new log
I20260808 16:55:50.554032 4501 ts_tablet_manager.cc:1397] T ed17abdcf08940c3a3b1cb5a57a775e8 P b2b673c9a0b54c71b963c8a414dea61f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 16:55:50.554206 4502 tablet_bootstrap.cc:492] T ed17abdcf08940c3a3b1cb5a57a775e8 P 170adff766bf483d9ab90fba2f1e301c: Bootstrap starting.
I20260808 16:55:50.554541 4503 tablet_bootstrap.cc:492] T ed17abdcf08940c3a3b1cb5a57a775e8 P e2384c82f6c24e29952eca100df5819d: No bootstrap required, opened a new log
I20260808 16:55:50.554602 4503 ts_tablet_manager.cc:1397] T ed17abdcf08940c3a3b1cb5a57a775e8 P e2384c82f6c24e29952eca100df5819d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 16:55:50.554869 4502 tablet_bootstrap.cc:654] T ed17abdcf08940c3a3b1cb5a57a775e8 P 170adff766bf483d9ab90fba2f1e301c: Neither blocks nor log segments found. Creating new log.
I20260808 16:55:50.555215 4502 log.cc:824] T ed17abdcf08940c3a3b1cb5a57a775e8 P 170adff766bf483d9ab90fba2f1e301c: Log is configured to *not* fsync() on all Append() calls
I20260808 16:55:50.555972 4502 tablet_bootstrap.cc:492] T ed17abdcf08940c3a3b1cb5a57a775e8 P 170adff766bf483d9ab90fba2f1e301c: No bootstrap required, opened a new log
I20260808 16:55:50.556084 4502 ts_tablet_manager.cc:1397] T ed17abdcf08940c3a3b1cb5a57a775e8 P 170adff766bf483d9ab90fba2f1e301c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 16:55:50.556442 4501 raft_consensus.cc:348] T ed17abdcf08940c3a3b1cb5a57a775e8 P b2b673c9a0b54c71b963c8a414dea61f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2384c82f6c24e29952eca100df5819d" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 35259 } } peers { permanent_uuid: "b2b673c9a0b54c71b963c8a414dea61f" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 45313 } } peers { permanent_uuid: "170adff766bf483d9ab90fba2f1e301c" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 45039 } }
I20260808 16:55:50.556599 4501 raft_consensus.cc:374] T ed17abdcf08940c3a3b1cb5a57a775e8 P b2b673c9a0b54c71b963c8a414dea61f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:55:50.556636 4501 raft_consensus.cc:729] T ed17abdcf08940c3a3b1cb5a57a775e8 P b2b673c9a0b54c71b963c8a414dea61f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2b673c9a0b54c71b963c8a414dea61f, State: Initialized, Role: FOLLOWER
I20260808 16:55:50.556593 4503 raft_consensus.cc:348] T ed17abdcf08940c3a3b1cb5a57a775e8 P e2384c82f6c24e29952eca100df5819d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2384c82f6c24e29952eca100df5819d" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 35259 } } peers { permanent_uuid: "b2b673c9a0b54c71b963c8a414dea61f" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 45313 } } peers { permanent_uuid: "170adff766bf483d9ab90fba2f1e301c" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 45039 } }
I20260808 16:55:50.556751 4503 raft_consensus.cc:374] T ed17abdcf08940c3a3b1cb5a57a775e8 P e2384c82f6c24e29952eca100df5819d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:55:50.556804 4503 raft_consensus.cc:729] T ed17abdcf08940c3a3b1cb5a57a775e8 P e2384c82f6c24e29952eca100df5819d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2384c82f6c24e29952eca100df5819d, State: Initialized, Role: FOLLOWER
I20260808 16:55:50.556833 4501 consensus_queue.cc:260] T ed17abdcf08940c3a3b1cb5a57a775e8 P b2b673c9a0b54c71b963c8a414dea61f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2384c82f6c24e29952eca100df5819d" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 35259 } } peers { permanent_uuid: "b2b673c9a0b54c71b963c8a414dea61f" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 45313 } } peers { permanent_uuid: "170adff766bf483d9ab90fba2f1e301c" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 45039 } }
I20260808 16:55:50.556914 4503 consensus_queue.cc:260] T ed17abdcf08940c3a3b1cb5a57a775e8 P e2384c82f6c24e29952eca100df5819d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2384c82f6c24e29952eca100df5819d" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 35259 } } peers { permanent_uuid: "b2b673c9a0b54c71b963c8a414dea61f" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 45313 } } peers { permanent_uuid: "170adff766bf483d9ab90fba2f1e301c" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 45039 } }
I20260808 16:55:50.557119 4501 ts_tablet_manager.cc:1428] T ed17abdcf08940c3a3b1cb5a57a775e8 P b2b673c9a0b54c71b963c8a414dea61f: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260808 16:55:50.557158 4503 ts_tablet_manager.cc:1428] T ed17abdcf08940c3a3b1cb5a57a775e8 P e2384c82f6c24e29952eca100df5819d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 16:55:50.557168 4279 heartbeater.cc:503] Master 127.8.24.62:37943 was elected leader, sending a full tablet report...
I20260808 16:55:50.557313 4093 heartbeater.cc:503] Master 127.8.24.62:37943 was elected leader, sending a full tablet report...
I20260808 16:55:50.558480 4502 raft_consensus.cc:348] T ed17abdcf08940c3a3b1cb5a57a775e8 P 170adff766bf483d9ab90fba2f1e301c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2384c82f6c24e29952eca100df5819d" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 35259 } } peers { permanent_uuid: "b2b673c9a0b54c71b963c8a414dea61f" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 45313 } } peers { permanent_uuid: "170adff766bf483d9ab90fba2f1e301c" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 45039 } }
I20260808 16:55:50.558609 4502 raft_consensus.cc:374] T ed17abdcf08940c3a3b1cb5a57a775e8 P 170adff766bf483d9ab90fba2f1e301c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:55:50.558629 4502 raft_consensus.cc:729] T ed17abdcf08940c3a3b1cb5a57a775e8 P 170adff766bf483d9ab90fba2f1e301c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 170adff766bf483d9ab90fba2f1e301c, State: Initialized, Role: FOLLOWER
I20260808 16:55:50.558727 4502 consensus_queue.cc:260] T ed17abdcf08940c3a3b1cb5a57a775e8 P 170adff766bf483d9ab90fba2f1e301c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2384c82f6c24e29952eca100df5819d" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 35259 } } peers { permanent_uuid: "b2b673c9a0b54c71b963c8a414dea61f" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 45313 } } peers { permanent_uuid: "170adff766bf483d9ab90fba2f1e301c" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 45039 } }
I20260808 16:55:50.558938 4502 ts_tablet_manager.cc:1428] T ed17abdcf08940c3a3b1cb5a57a775e8 P 170adff766bf483d9ab90fba2f1e301c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 16:55:50.558997 4477 heartbeater.cc:503] Master 127.8.24.62:37943 was elected leader, sending a full tablet report...
I20260808 16:55:50.594637 4509 raft_consensus.cc:482] T ed17abdcf08940c3a3b1cb5a57a775e8 P e2384c82f6c24e29952eca100df5819d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 16:55:50.594712 4509 raft_consensus.cc:504] T ed17abdcf08940c3a3b1cb5a57a775e8 P e2384c82f6c24e29952eca100df5819d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2384c82f6c24e29952eca100df5819d" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 35259 } } peers { permanent_uuid: "b2b673c9a0b54c71b963c8a414dea61f" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 45313 } } peers { permanent_uuid: "170adff766bf483d9ab90fba2f1e301c" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 45039 } }
I20260808 16:55:50.594964 4509 leader_election.cc:288] T ed17abdcf08940c3a3b1cb5a57a775e8 P e2384c82f6c24e29952eca100df5819d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b2b673c9a0b54c71b963c8a414dea61f (127.8.24.2:45313), 170adff766bf483d9ab90fba2f1e301c (127.8.24.3:45039)
I20260808 16:55:50.598232 4500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:50.595160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:60193 (local address 127.8.24.3:45039)
0808 16:55:50.595197 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 16:55:50.595199 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:55:50.595292 (+ 93us) server_negotiation.cc:407] Connection header received
0808 16:55:50.595340 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:55:50.595341 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:55:50.595347 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:55:50.595384 (+ 37us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:55:50.595490 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:50.596134 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:50.597391 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:50.597910 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:50.597972 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:50.597980 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 16:55:50.598138 (+ 158us) server_negotiation.cc:299] Negotiation successful
0808 16:55:50.598169 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 16:55:50.598248 4497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:50.595102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:45951 (local address 127.8.24.2:45313)
0808 16:55:50.595141 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 16:55:50.595143 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:55:50.595151 (+ 8us) server_negotiation.cc:407] Connection header received
0808 16:55:50.595227 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:55:50.595228 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:55:50.595235 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:55:50.595286 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:55:50.595414 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:50.596096 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:50.597391 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:50.597916 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:50.597976 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:50.597982 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0808 16:55:50.598138 (+ 156us) server_negotiation.cc:299] Negotiation successful
0808 16:55:50.598198 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 16:55:50.598258 4095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:50.595062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.2:45313 (local address 127.8.24.1:45951)
0808 16:55:50.595099 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:50.595108 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 16:55:50.595147 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:55:50.595301 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:55:50.595302 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:55:50.595303 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:55:50.595357 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:50.595359 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:50.596162 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:50.596163 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:50.597317 (+ 1154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:50.597320 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:50.597983 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:50.597984 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:50.598075 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:50.598080 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 16:55:50.598129 (+ 49us) client_negotiation.cc:240] Negotiation successful
0808 16:55:50.598169 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20,"mutex_wait_us":9}
I20260808 16:55:50.598265 4513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:50.595102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.3:45039 (local address 127.8.24.1:60193)
0808 16:55:50.595231 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:50.595240 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 16:55:50.595295 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:55:50.595387 (+ 92us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:55:50.595388 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:55:50.595389 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:55:50.595442 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:50.595443 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:50.596188 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:50.596188 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:50.597320 (+ 1132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:50.597321 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:50.597983 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:50.597984 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:50.598075 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:50.598080 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 16:55:50.598129 (+ 49us) client_negotiation.cc:240] Negotiation successful
0808 16:55:50.598168 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"mutex_wait_us":14,"thread_start_us":75,"threads_started":1}
I20260808 16:55:50.598593 4207 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed17abdcf08940c3a3b1cb5a57a775e8" candidate_uuid: "e2384c82f6c24e29952eca100df5819d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2b673c9a0b54c71b963c8a414dea61f" is_pre_election: true
I20260808 16:55:50.598662 4413 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed17abdcf08940c3a3b1cb5a57a775e8" candidate_uuid: "e2384c82f6c24e29952eca100df5819d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170adff766bf483d9ab90fba2f1e301c" is_pre_election: true
I20260808 16:55:50.598738 4207 raft_consensus.cc:2454] T ed17abdcf08940c3a3b1cb5a57a775e8 P b2b673c9a0b54c71b963c8a414dea61f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2384c82f6c24e29952eca100df5819d in term 0.
I20260808 16:55:50.598766 4413 raft_consensus.cc:2454] T ed17abdcf08940c3a3b1cb5a57a775e8 P 170adff766bf483d9ab90fba2f1e301c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2384c82f6c24e29952eca100df5819d in term 0.
I20260808 16:55:50.598946 3912 leader_election.cc:302] T ed17abdcf08940c3a3b1cb5a57a775e8 P e2384c82f6c24e29952eca100df5819d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170adff766bf483d9ab90fba2f1e301c, e2384c82f6c24e29952eca100df5819d; no voters:
I20260808 16:55:50.599048 4509 raft_consensus.cc:2781] T ed17abdcf08940c3a3b1cb5a57a775e8 P e2384c82f6c24e29952eca100df5819d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 16:55:50.599097 4509 raft_consensus.cc:482] T ed17abdcf08940c3a3b1cb5a57a775e8 P e2384c82f6c24e29952eca100df5819d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 16:55:50.599126 4509 raft_consensus.cc:3037] T ed17abdcf08940c3a3b1cb5a57a775e8 P e2384c82f6c24e29952eca100df5819d [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:55:50.599674 4509 raft_consensus.cc:504] T ed17abdcf08940c3a3b1cb5a57a775e8 P e2384c82f6c24e29952eca100df5819d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2384c82f6c24e29952eca100df5819d" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 35259 } } peers { permanent_uuid: "b2b673c9a0b54c71b963c8a414dea61f" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 45313 } } peers { permanent_uuid: "170adff766bf483d9ab90fba2f1e301c" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 45039 } }
I20260808 16:55:50.599781 4509 leader_election.cc:288] T ed17abdcf08940c3a3b1cb5a57a775e8 P e2384c82f6c24e29952eca100df5819d [CANDIDATE]: Term 1 election: Requested vote from peers b2b673c9a0b54c71b963c8a414dea61f (127.8.24.2:45313), 170adff766bf483d9ab90fba2f1e301c (127.8.24.3:45039)
I20260808 16:55:50.599926 4207 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed17abdcf08940c3a3b1cb5a57a775e8" candidate_uuid: "e2384c82f6c24e29952eca100df5819d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2b673c9a0b54c71b963c8a414dea61f"
I20260808 16:55:50.599924 4413 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed17abdcf08940c3a3b1cb5a57a775e8" candidate_uuid: "e2384c82f6c24e29952eca100df5819d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170adff766bf483d9ab90fba2f1e301c"
I20260808 16:55:50.599992 4413 raft_consensus.cc:3037] T ed17abdcf08940c3a3b1cb5a57a775e8 P 170adff766bf483d9ab90fba2f1e301c [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:55:50.599989 4207 raft_consensus.cc:3037] T ed17abdcf08940c3a3b1cb5a57a775e8 P b2b673c9a0b54c71b963c8a414dea61f [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:55:50.600602 4207 raft_consensus.cc:2454] T ed17abdcf08940c3a3b1cb5a57a775e8 P b2b673c9a0b54c71b963c8a414dea61f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2384c82f6c24e29952eca100df5819d in term 1.
I20260808 16:55:50.600613 4413 raft_consensus.cc:2454] T ed17abdcf08940c3a3b1cb5a57a775e8 P 170adff766bf483d9ab90fba2f1e301c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2384c82f6c24e29952eca100df5819d in term 1.
I20260808 16:55:50.600744 3912 leader_election.cc:302] T ed17abdcf08940c3a3b1cb5a57a775e8 P e2384c82f6c24e29952eca100df5819d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170adff766bf483d9ab90fba2f1e301c, e2384c82f6c24e29952eca100df5819d; no voters:
I20260808 16:55:50.600865 4509 raft_consensus.cc:2781] T ed17abdcf08940c3a3b1cb5a57a775e8 P e2384c82f6c24e29952eca100df5819d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:55:50.600972 4509 raft_consensus.cc:686] T ed17abdcf08940c3a3b1cb5a57a775e8 P e2384c82f6c24e29952eca100df5819d [term 1 LEADER]: Becoming Leader. State: Replica: e2384c82f6c24e29952eca100df5819d, State: Running, Role: LEADER
I20260808 16:55:50.601070 4509 consensus_queue.cc:237] T ed17abdcf08940c3a3b1cb5a57a775e8 P e2384c82f6c24e29952eca100df5819d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2384c82f6c24e29952eca100df5819d" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 35259 } } peers { permanent_uuid: "b2b673c9a0b54c71b963c8a414dea61f" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 45313 } } peers { permanent_uuid: "170adff766bf483d9ab90fba2f1e301c" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 45039 } }
I20260808 16:55:50.601735 3792 catalog_manager.cc:5495] T ed17abdcf08940c3a3b1cb5a57a775e8 P e2384c82f6c24e29952eca100df5819d reported cstate change: term changed from 0 to 1, leader changed from <none> to e2384c82f6c24e29952eca100df5819d (127.8.24.1). New cstate: current_term: 1 leader_uuid: "e2384c82f6c24e29952eca100df5819d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2384c82f6c24e29952eca100df5819d" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 35259 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b2b673c9a0b54c71b963c8a414dea61f" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 45313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170adff766bf483d9ab90fba2f1e301c" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 45039 } health_report { overall_health: UNKNOWN } } }
I20260808 16:55:50.608631 4494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:50.605935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49742 (local address 127.8.24.1:35259)
0808 16:55:50.605972 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 16:55:50.605974 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:55:50.605980 (+ 6us) server_negotiation.cc:407] Connection header received
0808 16:55:50.606037 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:55:50.606037 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:55:50.606053 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:55:50.606111 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 16:55:50.606228 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:50.606723 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:50.607750 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:50.608045 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:50.608109 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:50.608318 (+ 209us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 16:55:50.608425 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 16:55:50.608514 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 16:55:50.608530 (+ 16us) server_negotiation.cc:299] Negotiation successful
0808 16:55:50.608549 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 16:55:50.610316 4413 raft_consensus.cc:1261] T ed17abdcf08940c3a3b1cb5a57a775e8 P 170adff766bf483d9ab90fba2f1e301c [term 1 FOLLOWER]: Refusing update from remote peer e2384c82f6c24e29952eca100df5819d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:55:50.610574 4207 raft_consensus.cc:1261] T ed17abdcf08940c3a3b1cb5a57a775e8 P b2b673c9a0b54c71b963c8a414dea61f [term 1 FOLLOWER]: Refusing update from remote peer e2384c82f6c24e29952eca100df5819d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:55:50.610760 4514 consensus_queue.cc:1035] T ed17abdcf08940c3a3b1cb5a57a775e8 P e2384c82f6c24e29952eca100df5819d [LEADER]: Connected to new peer: Peer: permanent_uuid: "170adff766bf483d9ab90fba2f1e301c" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 45039 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:55:50.610881 4509 consensus_queue.cc:1035] T ed17abdcf08940c3a3b1cb5a57a775e8 P e2384c82f6c24e29952eca100df5819d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2b673c9a0b54c71b963c8a414dea61f" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 45313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:55:50.612823 4519 mvcc.cc:205] Tried to move back new op lower bound from 7316308584897146880 to 7316308584862556160. Current Snapshot: MvccSnapshot[applied={T|T < 7316308584897146880}]
I20260808 16:55:50.613042 4520 mvcc.cc:205] Tried to move back new op lower bound from 7316308584897146880 to 7316308584862556160. Current Snapshot: MvccSnapshot[applied={T|T < 7316308584897146880}]
I20260808 16:55:50.615154 3792 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:53598:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 16:55:50.615229 3792 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:53598:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 16:55:50.615810 3792 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 64c90fbab96f4a05ba11187842fb1b7d: Sending DeleteTablet for 3 replicas of tablet ed17abdcf08940c3a3b1cb5a57a775e8
I20260808 16:55:50.616113 4374 tablet_service.cc:1514] Processing DeleteTablet for tablet ed17abdcf08940c3a3b1cb5a57a775e8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:55:50 UTC) from {username='slave'} at 127.0.0.1:46852
I20260808 16:55:50.616113 4174 tablet_service.cc:1514] Processing DeleteTablet for tablet ed17abdcf08940c3a3b1cb5a57a775e8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:55:50 UTC) from {username='slave'} at 127.0.0.1:46032
I20260808 16:55:50.616318 4537 tablet_replica.cc:330] T ed17abdcf08940c3a3b1cb5a57a775e8 P b2b673c9a0b54c71b963c8a414dea61f: stopping tablet replica
I20260808 16:55:50.616329 3987 tablet_service.cc:1514] Processing DeleteTablet for tablet ed17abdcf08940c3a3b1cb5a57a775e8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:55:50 UTC) from {username='slave'} at 127.0.0.1:49730
I20260808 16:55:50.616411 4537 raft_consensus.cc:2229] T ed17abdcf08940c3a3b1cb5a57a775e8 P b2b673c9a0b54c71b963c8a414dea61f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 16:55:50.616503 4537 raft_consensus.cc:2258] T ed17abdcf08940c3a3b1cb5a57a775e8 P b2b673c9a0b54c71b963c8a414dea61f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:55:50.616550 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 3889
I20260808 16:55:50.629729 4536 tablet_replica.cc:330] T ed17abdcf08940c3a3b1cb5a57a775e8 P 170adff766bf483d9ab90fba2f1e301c: stopping tablet replica
I20260808 16:55:50.629803 4536 raft_consensus.cc:2229] T ed17abdcf08940c3a3b1cb5a57a775e8 P 170adff766bf483d9ab90fba2f1e301c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 16:55:50.629866 4536 raft_consensus.cc:2258] T ed17abdcf08940c3a3b1cb5a57a775e8 P 170adff766bf483d9ab90fba2f1e301c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:55:50.632808 4536 ts_tablet_manager.cc:1905] T ed17abdcf08940c3a3b1cb5a57a775e8 P 170adff766bf483d9ab90fba2f1e301c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:55:50.634413 4536 ts_tablet_manager.cc:1918] T ed17abdcf08940c3a3b1cb5a57a775e8 P 170adff766bf483d9ab90fba2f1e301c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 16:55:50.634475 4536 log.cc:1196] T ed17abdcf08940c3a3b1cb5a57a775e8 P 170adff766bf483d9ab90fba2f1e301c: Deleting WAL directory at /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786207919384338-8288-0/minicluster-data/ts-2/wal/wals/ed17abdcf08940c3a3b1cb5a57a775e8
I20260808 16:55:50.634660 4536 ts_tablet_manager.cc:1939] T ed17abdcf08940c3a3b1cb5a57a775e8 P 170adff766bf483d9ab90fba2f1e301c: Deleting consensus metadata
I20260808 16:55:50.635221 3763 catalog_manager.cc:4841] TS 170adff766bf483d9ab90fba2f1e301c (127.8.24.3:45039): tablet ed17abdcf08940c3a3b1cb5a57a775e8 (table test-table [id=596f722e997540649d200d33479c6183]) successfully deleted
I20260808 16:55:50.635304 4537 ts_tablet_manager.cc:1905] T ed17abdcf08940c3a3b1cb5a57a775e8 P b2b673c9a0b54c71b963c8a414dea61f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:55:50.636348 4537 ts_tablet_manager.cc:1918] T ed17abdcf08940c3a3b1cb5a57a775e8 P b2b673c9a0b54c71b963c8a414dea61f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 16:55:50.636411 4537 log.cc:1196] T ed17abdcf08940c3a3b1cb5a57a775e8 P b2b673c9a0b54c71b963c8a414dea61f: Deleting WAL directory at /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786207919384338-8288-0/minicluster-data/ts-1/wal/wals/ed17abdcf08940c3a3b1cb5a57a775e8
I20260808 16:55:50.636600 4537 ts_tablet_manager.cc:1939] T ed17abdcf08940c3a3b1cb5a57a775e8 P b2b673c9a0b54c71b963c8a414dea61f: Deleting consensus metadata
I20260808 16:55:50.636941 3764 catalog_manager.cc:4841] TS b2b673c9a0b54c71b963c8a414dea61f (127.8.24.2:45313): tablet ed17abdcf08940c3a3b1cb5a57a775e8 (table test-table [id=596f722e997540649d200d33479c6183]) successfully deleted
W20260808 16:55:50.639022 3764 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.24.1:35259 (error 108)
I20260808 16:55:50.639194 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 4096
I20260808 16:55:50.640527 4493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:50.639340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:35259 (local address 127.0.0.1:49744)
0808 16:55:50.640457 (+ 1117us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:50.640501 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.1:35259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1098}
W20260808 16:55:50.642982 3764 catalog_manager.cc:4584] TS e2384c82f6c24e29952eca100df5819d (127.8.24.1:35259): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ed17abdcf08940c3a3b1cb5a57a775e8: Network error: Client connection negotiation failed: client connection to 127.8.24.1:35259: connect: Connection refused (error 111)
I20260808 16:55:50.648205 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 4283
I20260808 16:55:50.656332 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 3731
2026-08-08T16:55:50Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (681 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786207919384338-8288-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 16:55:50 dist-test-slave-6zbq krb5kdc[4543](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 16:55:50 dist-test-slave-6zbq krb5kdc[4543](info): set up 2 sockets
Aug 08 16:55:50 dist-test-slave-6zbq krb5kdc[4543](info): commencing operation
krb5kdc: starting...
W20260808 16:55:51.788954 8288 mini_kdc.cc:121] Time spent starting KDC: real 1.107s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786207919384338-8288-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786207919384338-8288-0/krb5kdc/test-user.keytab.
Aug 08 16:55:51 dist-test-slave-6zbq krb5kdc[4543](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208151, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T16:55:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T16:55:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.24.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
I20260808 16:55:51.894740 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786207919384338-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786207919384338-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786207919384338-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:38209
--webserver_interface=127.8.24.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:37591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:38209
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:55:51.971763 4561 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:55:51.971933 4561 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:55:51.971953 4561 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:55:51.973033 4561 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 16:55:51.973071 4561 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:55:51.973084 4561 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:55:51.973107 4561 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 16:55:51.973131 4561 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 16:55:51.974592 4561 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:37591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786207919384338-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786207919384338-8288-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:38209
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:38209
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.24.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4561
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786207919384338-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:55:51.974814 4561 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:55:51.975011 4561 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:55:51.979274 4575 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:55:51.979321 4582 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:55:51.979295 4576 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:55:51.979496 4561 server_base.cc:1034] running on GCE node
I20260808 16:55:51.979751 4561 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:55:51.979990 4561 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:55:51.981163 4561 hybrid_clock.cc:648] HybridClock initialized: now 1786208151981149 us; error 27 us; skew 500 ppm
Aug 08 16:55:51 dist-test-slave-6zbq krb5kdc[4543](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208151, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:55:51.983385 4561 init.cc:377] Logged in from keytab as kudu/127.8.24.62@KRBTEST.COM (short username kudu)
I20260808 16:55:51.983770 4561 webserver.cc:460] Webserver started at http://127.8.24.62:46551/ using document root <none> and password file <none>
I20260808 16:55:51.983973 4561 fs_manager.cc:359] Metadata directory not provided
I20260808 16:55:51.984019 4561 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:55:51.984140 4561 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:55:51.985069 4561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786207919384338-8288-0/minicluster-data/master-0/data/instance:
uuid: "c390815c100f47a19114e87e07c509fd"
format_stamp: "Formatted at 2026-08-08 16:55:51 on dist-test-slave-6zbq"
server_key: "909b75d5a8314f99cf51bfd3358c8511"
server_key_iv: "962862d166bd533d3d3cabb64e20bfa0"
server_key_version: "encryptionkey@0"
I20260808 16:55:51.985374 4561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786207919384338-8288-0/minicluster-data/master-0/wal/instance:
uuid: "c390815c100f47a19114e87e07c509fd"
format_stamp: "Formatted at 2026-08-08 16:55:51 on dist-test-slave-6zbq"
server_key: "909b75d5a8314f99cf51bfd3358c8511"
server_key_iv: "962862d166bd533d3d3cabb64e20bfa0"
server_key_version: "encryptionkey@0"
I20260808 16:55:51.986567 4561 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 16:55:51.987226 4594 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:55:51.987377 4561 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 16:55:51.987457 4561 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786207919384338-8288-0/minicluster-data/master-0/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786207919384338-8288-0/minicluster-data/master-0/wal
uuid: "c390815c100f47a19114e87e07c509fd"
format_stamp: "Formatted at 2026-08-08 16:55:51 on dist-test-slave-6zbq"
server_key: "909b75d5a8314f99cf51bfd3358c8511"
server_key_iv: "962862d166bd533d3d3cabb64e20bfa0"
server_key_version: "encryptionkey@0"
I20260808 16:55:51.987514 4561 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786207919384338-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786207919384338-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786207919384338-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:55:52.010560 4561 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:55:52.012403 4561 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:55:52.012529 4561 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:55:52.016072 4561 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.62:38209
I20260808 16:55:52.016106 4672 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.62:38209 every 8 connection(s)
I20260808 16:55:52.016402 4561 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
I20260808 16:55:52.017201 4673 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:55:52.018532 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 4561
I20260808 16:55:52.018637 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786207919384338-8288-0/minicluster-data/master-0/wal/instance
I20260808 16:55:52.018821 8288 external_mini_cluster.cc:1428] Setting key bab15fff821b65b3e57b95f91fa6af3b
I20260808 16:55:52.019569 4673 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c390815c100f47a19114e87e07c509fd: Bootstrap starting.
I20260808 16:55:52.020234 4673 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c390815c100f47a19114e87e07c509fd: Neither blocks nor log segments found. Creating new log.
I20260808 16:55:52.020512 4673 log.cc:824] T 00000000000000000000000000000000 P c390815c100f47a19114e87e07c509fd: Log is configured to *not* fsync() on all Append() calls
I20260808 16:55:52.021251 4673 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c390815c100f47a19114e87e07c509fd: No bootstrap required, opened a new log
I20260808 16:55:52.023413 4673 raft_consensus.cc:348] T 00000000000000000000000000000000 P c390815c100f47a19114e87e07c509fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c390815c100f47a19114e87e07c509fd" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 38209 } }
I20260808 16:55:52.023572 4673 raft_consensus.cc:374] T 00000000000000000000000000000000 P c390815c100f47a19114e87e07c509fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:55:52.023620 4673 raft_consensus.cc:729] T 00000000000000000000000000000000 P c390815c100f47a19114e87e07c509fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c390815c100f47a19114e87e07c509fd, State: Initialized, Role: FOLLOWER
I20260808 16:55:52.023752 4673 consensus_queue.cc:260] T 00000000000000000000000000000000 P c390815c100f47a19114e87e07c509fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c390815c100f47a19114e87e07c509fd" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 38209 } }
I20260808 16:55:52.023824 4673 raft_consensus.cc:388] T 00000000000000000000000000000000 P c390815c100f47a19114e87e07c509fd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 16:55:52.023855 4673 raft_consensus.cc:482] T 00000000000000000000000000000000 P c390815c100f47a19114e87e07c509fd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 16:55:52.023900 4673 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c390815c100f47a19114e87e07c509fd [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:55:52.024537 4673 raft_consensus.cc:504] T 00000000000000000000000000000000 P c390815c100f47a19114e87e07c509fd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c390815c100f47a19114e87e07c509fd" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 38209 } }
I20260808 16:55:52.024653 4673 leader_election.cc:302] T 00000000000000000000000000000000 P c390815c100f47a19114e87e07c509fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c390815c100f47a19114e87e07c509fd; no voters:
I20260808 16:55:52.024809 4673 leader_election.cc:288] T 00000000000000000000000000000000 P c390815c100f47a19114e87e07c509fd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 16:55:52.024849 4679 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c390815c100f47a19114e87e07c509fd [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:55:52.024983 4679 raft_consensus.cc:686] T 00000000000000000000000000000000 P c390815c100f47a19114e87e07c509fd [term 1 LEADER]: Becoming Leader. State: Replica: c390815c100f47a19114e87e07c509fd, State: Running, Role: LEADER
I20260808 16:55:52.025058 4673 sys_catalog.cc:564] T 00000000000000000000000000000000 P c390815c100f47a19114e87e07c509fd [sys.catalog]: configured and running, proceeding with master startup.
I20260808 16:55:52.025076 4679 consensus_queue.cc:237] T 00000000000000000000000000000000 P c390815c100f47a19114e87e07c509fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c390815c100f47a19114e87e07c509fd" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 38209 } }
I20260808 16:55:52.025386 4679 sys_catalog.cc:455] T 00000000000000000000000000000000 P c390815c100f47a19114e87e07c509fd [sys.catalog]: SysCatalogTable state changed. Reason: New leader c390815c100f47a19114e87e07c509fd. Latest consensus state: current_term: 1 leader_uuid: "c390815c100f47a19114e87e07c509fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c390815c100f47a19114e87e07c509fd" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 38209 } } }
I20260808 16:55:52.025475 4679 sys_catalog.cc:458] T 00000000000000000000000000000000 P c390815c100f47a19114e87e07c509fd [sys.catalog]: This master's current role is: LEADER
I20260808 16:55:52.025398 4682 sys_catalog.cc:455] T 00000000000000000000000000000000 P c390815c100f47a19114e87e07c509fd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c390815c100f47a19114e87e07c509fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c390815c100f47a19114e87e07c509fd" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 38209 } } }
I20260808 16:55:52.025601 4682 sys_catalog.cc:458] T 00000000000000000000000000000000 P c390815c100f47a19114e87e07c509fd [sys.catalog]: This master's current role is: LEADER
Aug 08 16:55:52 dist-test-slave-6zbq krb5kdc[4543](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208151, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260808 16:55:52.025673 4689 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 16:55:52.026101 4689 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 16:55:52.027832 4689 catalog_manager.cc:1347] Generated new cluster ID: 06a721905a964318a0e95c157423d8ce
I20260808 16:55:52.027885 4689 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 16:55:52.029237 4677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:52.019251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51368 (local address 127.8.24.62:38209)
0808 16:55:52.019428 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0808 16:55:52.019437 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 16:55:52.019467 (+ 30us) server_negotiation.cc:407] Connection header received
0808 16:55:52.019994 (+ 527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:55:52.020002 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:55:52.020057 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:55:52.020202 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:55:52.020839 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:52.022151 (+ 1312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:52.022914 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:52.023512 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:52.023628 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:52.026175 (+ 2547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:55:52.026180 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:55:52.026180 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:55:52.026191 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:55:52.027885 (+ 1694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:55:52.028198 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:55:52.028201 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:55:52.028201 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:55:52.028286 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:55:52.028448 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:55:52.028448 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:55:52.028449 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:55:52.028661 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:55:52.028782 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0808 16:55:52.028897 (+ 115us) server_negotiation.cc:299] Negotiation successful
0808 16:55:52.029022 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":74,"threads_started":1}
I20260808 16:55:52.063081 4689 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 16:55:52.063756 4689 catalog_manager.cc:1504] Loading token signing keys...
I20260808 16:55:52.069625 4689 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c390815c100f47a19114e87e07c509fd: Generated new TSK 0
I20260808 16:55:52.069792 4689 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.24.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.1.keytab.
Entry for principal kudu/127.8.24.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.1.keytab.
I20260808 16:55:52.095211 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786207919384338-8288-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786207919384338-8288-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786207919384338-8288-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786207919384338-8288-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.1:0
--local_ip_for_outbound_sockets=127.8.24.1
--webserver_interface=127.8.24.1
--webserver_port=0
--tserver_master_addrs=127.8.24.62:38209
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.1.keytab
--principal=kudu/127.8.24.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:37591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:55:52.173000 4709 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:55:52.173177 4709 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:55:52.173197 4709 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:55:52.174288 4709 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:55:52.174326 4709 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:55:52.174391 4709 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.1
I20260808 16:55:52.175787 4709 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:37591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786207919384338-8288-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786207919384338-8288-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.1.keytab
--principal=kudu/127.8.24.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786207919384338-8288-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.24.1
--webserver_port=0
--tserver_master_addrs=127.8.24.62:38209
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4709
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.1
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786207919384338-8288-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:55:52.175987 4709 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:55:52.176182 4709 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:55:52.180655 4718 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:55:52.180779 4709 server_base.cc:1034] running on GCE node
W20260808 16:55:52.180672 4716 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:55:52.180665 4721 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:55:52.181058 4709 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:55:52.181286 4709 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:55:52.182394 4709 hybrid_clock.cc:648] HybridClock initialized: now 1786208152182378 us; error 28 us; skew 500 ppm
Aug 08 16:55:52 dist-test-slave-6zbq krb5kdc[4543](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208152, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:55:52.184854 4709 init.cc:377] Logged in from keytab as kudu/127.8.24.1@KRBTEST.COM (short username kudu)
I20260808 16:55:52.185241 4709 webserver.cc:460] Webserver started at http://127.8.24.1:46873/ using document root <none> and password file <none>
I20260808 16:55:52.185449 4709 fs_manager.cc:359] Metadata directory not provided
I20260808 16:55:52.185497 4709 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:55:52.185618 4709 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:55:52.186606 4709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786207919384338-8288-0/minicluster-data/ts-0/data/instance:
uuid: "1fa6734033b542419d2801bbf29187a9"
format_stamp: "Formatted at 2026-08-08 16:55:52 on dist-test-slave-6zbq"
server_key: "9069631631d097326fda2e0ffbfc2ed4"
server_key_iv: "6b04a1b7cd6da3f067e8e2015c652142"
server_key_version: "encryptionkey@0"
I20260808 16:55:52.186914 4709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786207919384338-8288-0/minicluster-data/ts-0/wal/instance:
uuid: "1fa6734033b542419d2801bbf29187a9"
format_stamp: "Formatted at 2026-08-08 16:55:52 on dist-test-slave-6zbq"
server_key: "9069631631d097326fda2e0ffbfc2ed4"
server_key_iv: "6b04a1b7cd6da3f067e8e2015c652142"
server_key_version: "encryptionkey@0"
I20260808 16:55:52.188046 4709 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 16:55:52.188654 4732 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:55:52.188786 4709 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 16:55:52.188853 4709 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786207919384338-8288-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786207919384338-8288-0/minicluster-data/ts-0/wal
uuid: "1fa6734033b542419d2801bbf29187a9"
format_stamp: "Formatted at 2026-08-08 16:55:52 on dist-test-slave-6zbq"
server_key: "9069631631d097326fda2e0ffbfc2ed4"
server_key_iv: "6b04a1b7cd6da3f067e8e2015c652142"
server_key_version: "encryptionkey@0"
I20260808 16:55:52.188917 4709 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786207919384338-8288-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786207919384338-8288-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786207919384338-8288-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:55:52.202579 4709 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:55:52.204033 4709 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:55:52.204162 4709 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:55:52.204377 4709 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:55:52.204680 4709 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:55:52.204715 4709 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:55:52.204741 4709 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:55:52.204757 4709 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:55:52.215157 4709 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.1:44387
I20260808 16:55:52.215173 4911 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.1:44387 every 8 connection(s)
I20260808 16:55:52.215519 4709 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786207919384338-8288-0/minicluster-data/ts-0/data/info.pb
I20260808 16:55:52.219365 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 4709
I20260808 16:55:52.219471 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786207919384338-8288-0/minicluster-data/ts-0/wal/instance
I20260808 16:55:52.219648 8288 external_mini_cluster.cc:1428] Setting key ba43493c1bfabd1845f00425d1d604fe
Aug 08 16:55:52 dist-test-slave-6zbq krb5kdc[4543](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208152, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.1@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260808 16:55:52.224423 4677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:52.216413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:35599 (local address 127.8.24.62:38209)
0808 16:55:52.216458 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 16:55:52.216460 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:55:52.216469 (+ 9us) server_negotiation.cc:407] Connection header received
0808 16:55:52.216923 (+ 454us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:55:52.216925 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:55:52.216937 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:55:52.216989 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:55:52.217830 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:52.218323 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:52.219295 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:52.219527 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:52.219581 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:52.221804 (+ 2223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:55:52.221805 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:55:52.221805 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:55:52.221812 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:55:52.223176 (+ 1364us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:55:52.223604 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:55:52.223604 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:55:52.223605 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:55:52.223653 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:55:52.223886 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:55:52.223886 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:55:52.223887 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:55:52.223977 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:55:52.224021 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0808 16:55:52.224299 (+ 278us) server_negotiation.cc:299] Negotiation successful
0808 16:55:52.224362 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 16:55:52.224681 4916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:52.216127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:38209 (local address 127.8.24.1:35599)
0808 16:55:52.216309 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:52.216342 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 16:55:52.216832 (+ 490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:55:52.217059 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:55:52.217063 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:55:52.217568 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:55:52.217744 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:52.217748 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:52.218429 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:52.218430 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:52.219220 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:52.219223 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:52.219601 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:52.219602 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:52.219722 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:52.220178 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:55:52.220196 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:55:52.221720 (+ 1524us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:55:52.223264 (+ 1544us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:55:52.223272 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:55:52.223274 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:55:52.223537 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:55:52.223704 (+ 167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:55:52.223704 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:55:52.223705 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:55:52.223833 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:55:52.224027 (+ 194us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:55:52.224032 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:55:52.224181 (+ 149us) client_negotiation.cc:769] Sending connection context
0808 16:55:52.224298 (+ 117us) client_negotiation.cc:240] Negotiation successful
0808 16:55:52.224457 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":81,"threads_started":1}
I20260808 16:55:52.225135 4913 heartbeater.cc:347] Connected to a master server at 127.8.24.62:38209
I20260808 16:55:52.225222 4913 heartbeater.cc:464] Registering TS with master...
I20260808 16:55:52.225382 4913 heartbeater.cc:511] Master 127.8.24.62:38209 requested a full tablet report, sending...
I20260808 16:55:52.225868 4622 ts_manager.cc:194] Registered new tserver with Master: 1fa6734033b542419d2801bbf29187a9 (127.8.24.1:44387)
I20260808 16:55:52.226490 4622 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.1@KRBTEST.COM'} at 127.8.24.1:35599
WARNING: no policy specified for kudu/127.8.24.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.2.keytab.
Entry for principal kudu/127.8.24.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.2.keytab.
I20260808 16:55:52.242436 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786207919384338-8288-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786207919384338-8288-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786207919384338-8288-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786207919384338-8288-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.2:0
--local_ip_for_outbound_sockets=127.8.24.2
--webserver_interface=127.8.24.2
--webserver_port=0
--tserver_master_addrs=127.8.24.62:38209
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.2.keytab
--principal=kudu/127.8.24.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:37591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:55:52.320751 4920 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:55:52.320931 4920 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:55:52.320952 4920 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:55:52.322059 4920 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:55:52.322099 4920 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:55:52.322125 4920 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.2
I20260808 16:55:52.323578 4920 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:37591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786207919384338-8288-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786207919384338-8288-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.2.keytab
--principal=kudu/127.8.24.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786207919384338-8288-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.24.2
--webserver_port=0
--tserver_master_addrs=127.8.24.62:38209
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4920
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.2
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786207919384338-8288-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:55:52.323788 4920 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:55:52.323989 4920 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:55:52.328682 4928 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:55:52.328661 4931 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:55:52.328754 4920 server_base.cc:1034] running on GCE node
W20260808 16:55:52.328661 4927 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:55:52.329095 4920 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:55:52.329335 4920 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:55:52.330462 4920 hybrid_clock.cc:648] HybridClock initialized: now 1786208152330448 us; error 26 us; skew 500 ppm
Aug 08 16:55:52 dist-test-slave-6zbq krb5kdc[4543](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208152, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:55:52.332765 4920 init.cc:377] Logged in from keytab as kudu/127.8.24.2@KRBTEST.COM (short username kudu)
I20260808 16:55:52.333173 4920 webserver.cc:460] Webserver started at http://127.8.24.2:36293/ using document root <none> and password file <none>
I20260808 16:55:52.333364 4920 fs_manager.cc:359] Metadata directory not provided
I20260808 16:55:52.333411 4920 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:55:52.333534 4920 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:55:52.334533 4920 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786207919384338-8288-0/minicluster-data/ts-1/data/instance:
uuid: "3381e7c719804824b7adb2bd4fc47d3a"
format_stamp: "Formatted at 2026-08-08 16:55:52 on dist-test-slave-6zbq"
server_key: "f732a0fff49958e7fa527117dda88992"
server_key_iv: "fb3383c4af1c7de69db0a173939a203c"
server_key_version: "encryptionkey@0"
I20260808 16:55:52.334852 4920 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786207919384338-8288-0/minicluster-data/ts-1/wal/instance:
uuid: "3381e7c719804824b7adb2bd4fc47d3a"
format_stamp: "Formatted at 2026-08-08 16:55:52 on dist-test-slave-6zbq"
server_key: "f732a0fff49958e7fa527117dda88992"
server_key_iv: "fb3383c4af1c7de69db0a173939a203c"
server_key_version: "encryptionkey@0"
I20260808 16:55:52.335996 4920 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 16:55:52.336637 4944 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:55:52.336768 4920 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 16:55:52.336848 4920 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786207919384338-8288-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786207919384338-8288-0/minicluster-data/ts-1/wal
uuid: "3381e7c719804824b7adb2bd4fc47d3a"
format_stamp: "Formatted at 2026-08-08 16:55:52 on dist-test-slave-6zbq"
server_key: "f732a0fff49958e7fa527117dda88992"
server_key_iv: "fb3383c4af1c7de69db0a173939a203c"
server_key_version: "encryptionkey@0"
I20260808 16:55:52.336910 4920 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786207919384338-8288-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786207919384338-8288-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786207919384338-8288-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:55:52.346288 4920 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:55:52.348244 4920 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:55:52.348373 4920 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:55:52.348564 4920 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:55:52.348831 4920 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:55:52.348863 4920 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:55:52.348883 4920 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:55:52.348906 4920 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:55:52.354009 4920 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.2:43603
I20260808 16:55:52.354069 5112 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.2:43603 every 8 connection(s)
I20260808 16:55:52.354323 4920 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786207919384338-8288-0/minicluster-data/ts-1/data/info.pb
I20260808 16:55:52.356457 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 4920
I20260808 16:55:52.356565 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786207919384338-8288-0/minicluster-data/ts-1/wal/instance
I20260808 16:55:52.356748 8288 external_mini_cluster.cc:1428] Setting key dd188ad5deb372cdd0785b3df782a3b8
Aug 08 16:55:52 dist-test-slave-6zbq krb5kdc[4543](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208152, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.2@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260808 16:55:52.363024 4677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:52.354967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:60733 (local address 127.8.24.62:38209)
0808 16:55:52.355017 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 16:55:52.355019 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:55:52.355183 (+ 164us) server_negotiation.cc:407] Connection header received
0808 16:55:52.355664 (+ 481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:55:52.355666 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:55:52.355679 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:55:52.355729 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:55:52.356608 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:52.357202 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:52.358162 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:52.358393 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:52.358462 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:52.360422 (+ 1960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:55:52.360423 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:55:52.360423 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:55:52.360429 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:55:52.361693 (+ 1264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:55:52.362136 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:55:52.362136 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:55:52.362137 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:55:52.362180 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:55:52.362470 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:55:52.362470 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:55:52.362471 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:55:52.362566 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:55:52.362614 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 16:55:52.362905 (+ 291us) server_negotiation.cc:299] Negotiation successful
0808 16:55:52.362962 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 16:55:52.363265 5116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:52.354923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:38209 (local address 127.8.24.2:60733)
0808 16:55:52.355061 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:52.355103 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 16:55:52.355562 (+ 459us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:55:52.355792 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:55:52.355796 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:55:52.356327 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:55:52.356503 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:52.356507 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:52.357319 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:52.357320 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:52.358075 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:52.358076 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:52.358468 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:52.358470 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:52.358595 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:52.359011 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:55:52.359020 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:55:52.360338 (+ 1318us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:55:52.361778 (+ 1440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:55:52.361785 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:55:52.361787 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:55:52.362065 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:55:52.362239 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:55:52.362239 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:55:52.362240 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:55:52.362406 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:55:52.362624 (+ 218us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:55:52.362629 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:55:52.362782 (+ 153us) client_negotiation.cc:769] Sending connection context
0808 16:55:52.362897 (+ 115us) client_negotiation.cc:240] Negotiation successful
0808 16:55:52.363055 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":69,"threads_started":1}
I20260808 16:55:52.363689 5114 heartbeater.cc:347] Connected to a master server at 127.8.24.62:38209
I20260808 16:55:52.363775 5114 heartbeater.cc:464] Registering TS with master...
I20260808 16:55:52.363931 5114 heartbeater.cc:511] Master 127.8.24.62:38209 requested a full tablet report, sending...
I20260808 16:55:52.364308 4622 ts_manager.cc:194] Registered new tserver with Master: 3381e7c719804824b7adb2bd4fc47d3a (127.8.24.2:43603)
I20260808 16:55:52.364665 4622 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.2@KRBTEST.COM'} at 127.8.24.2:60733
WARNING: no policy specified for kudu/127.8.24.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.3.keytab.
Entry for principal kudu/127.8.24.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.3.keytab.
I20260808 16:55:52.380772 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786207919384338-8288-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786207919384338-8288-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786207919384338-8288-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786207919384338-8288-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.3:0
--local_ip_for_outbound_sockets=127.8.24.3
--webserver_interface=127.8.24.3
--webserver_port=0
--tserver_master_addrs=127.8.24.62:38209
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.3.keytab
--principal=kudu/127.8.24.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:37591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:55:52.458917 5119 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:55:52.459096 5119 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:55:52.459126 5119 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:55:52.460230 5119 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:55:52.460269 5119 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:55:52.460305 5119 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.3
I20260808 16:55:52.461720 5119 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:37591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786207919384338-8288-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786207919384338-8288-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.3.keytab
--principal=kudu/127.8.24.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786207919384338-8288-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.24.3
--webserver_port=0
--tserver_master_addrs=127.8.24.62:38209
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5119
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.3
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786207919384338-8288-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:55:52.461936 5119 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:55:52.462145 5119 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:55:52.466594 5125 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:55:52.466588 5124 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:55:52.466739 5119 server_base.cc:1034] running on GCE node
W20260808 16:55:52.466589 5127 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:55:52.466953 5119 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:55:52.467165 5119 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:55:52.468294 5119 hybrid_clock.cc:648] HybridClock initialized: now 1786208152468279 us; error 28 us; skew 500 ppm
Aug 08 16:55:52 dist-test-slave-6zbq krb5kdc[4543](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208152, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:55:52.470538 5119 init.cc:377] Logged in from keytab as kudu/127.8.24.3@KRBTEST.COM (short username kudu)
I20260808 16:55:52.470956 5119 webserver.cc:460] Webserver started at http://127.8.24.3:46351/ using document root <none> and password file <none>
I20260808 16:55:52.471171 5119 fs_manager.cc:359] Metadata directory not provided
I20260808 16:55:52.471217 5119 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:55:52.471338 5119 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:55:52.472262 5119 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786207919384338-8288-0/minicluster-data/ts-2/data/instance:
uuid: "2d4e5c451fb74bef8a43e69f5c480ac0"
format_stamp: "Formatted at 2026-08-08 16:55:52 on dist-test-slave-6zbq"
server_key: "b6ce3e16698d74e5fa26c1e5f5346d6f"
server_key_iv: "33921adb622fc45af8a6a64361a14c4e"
server_key_version: "encryptionkey@0"
I20260808 16:55:52.472572 5119 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786207919384338-8288-0/minicluster-data/ts-2/wal/instance:
uuid: "2d4e5c451fb74bef8a43e69f5c480ac0"
format_stamp: "Formatted at 2026-08-08 16:55:52 on dist-test-slave-6zbq"
server_key: "b6ce3e16698d74e5fa26c1e5f5346d6f"
server_key_iv: "33921adb622fc45af8a6a64361a14c4e"
server_key_version: "encryptionkey@0"
I20260808 16:55:52.473743 5119 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 16:55:52.474418 5140 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:55:52.474556 5119 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 16:55:52.474637 5119 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786207919384338-8288-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786207919384338-8288-0/minicluster-data/ts-2/wal
uuid: "2d4e5c451fb74bef8a43e69f5c480ac0"
format_stamp: "Formatted at 2026-08-08 16:55:52 on dist-test-slave-6zbq"
server_key: "b6ce3e16698d74e5fa26c1e5f5346d6f"
server_key_iv: "33921adb622fc45af8a6a64361a14c4e"
server_key_version: "encryptionkey@0"
I20260808 16:55:52.474702 5119 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786207919384338-8288-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786207919384338-8288-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786207919384338-8288-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:55:52.503324 5119 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:55:52.505033 5119 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:55:52.505160 5119 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:55:52.505363 5119 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:55:52.505664 5119 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:55:52.505698 5119 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:55:52.505728 5119 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:55:52.505744 5119 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:55:52.510991 5119 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.3:37935
I20260808 16:55:52.511047 5343 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.3:37935 every 8 connection(s)
I20260808 16:55:52.511320 5119 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786207919384338-8288-0/minicluster-data/ts-2/data/info.pb
I20260808 16:55:52.515416 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 5119
I20260808 16:55:52.515532 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786207919384338-8288-0/minicluster-data/ts-2/wal/instance
I20260808 16:55:52.515931 8288 external_mini_cluster.cc:1428] Setting key 9ce4143c43a75ecfd00cebcfdf1e4745
Aug 08 16:55:52 dist-test-slave-6zbq krb5kdc[4543](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208152, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.3@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260808 16:55:52.519586 4677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:52.511963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:32813 (local address 127.8.24.62:38209)
0808 16:55:52.512015 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 16:55:52.512017 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:55:52.512192 (+ 175us) server_negotiation.cc:407] Connection header received
0808 16:55:52.512644 (+ 452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:55:52.512645 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:55:52.512654 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:55:52.512691 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:55:52.513555 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:52.514053 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:52.515041 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:52.515260 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:52.515334 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:52.517358 (+ 2024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:55:52.517359 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:55:52.517359 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:55:52.517365 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:55:52.518596 (+ 1231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:55:52.518911 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:55:52.518912 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:55:52.518912 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:55:52.518957 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:55:52.519131 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:55:52.519132 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:55:52.519132 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:55:52.519234 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:55:52.519279 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0808 16:55:52.519470 (+ 191us) server_negotiation.cc:299] Negotiation successful
0808 16:55:52.519525 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 16:55:52.519729 5348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:52.511933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:38209 (local address 127.8.24.3:32813)
0808 16:55:52.512083 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:52.512123 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 16:55:52.512555 (+ 432us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:55:52.512749 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:55:52.512753 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:55:52.513296 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:55:52.513476 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:52.513481 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:52.514143 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:52.514145 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:52.514956 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:52.514959 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:52.515332 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:52.515333 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:52.515423 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:52.515811 (+ 388us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:55:52.515819 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:55:52.517281 (+ 1462us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:55:52.518651 (+ 1370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:55:52.518655 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:55:52.518656 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:55:52.518859 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:55:52.518999 (+ 140us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:55:52.519000 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:55:52.519000 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:55:52.519087 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:55:52.519277 (+ 190us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:55:52.519279 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:55:52.519380 (+ 101us) client_negotiation.cc:769] Sending connection context
0808 16:55:52.519451 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 16:55:52.519560 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":72,"threads_started":1}
I20260808 16:55:52.520114 5344 heartbeater.cc:347] Connected to a master server at 127.8.24.62:38209
I20260808 16:55:52.520210 5344 heartbeater.cc:464] Registering TS with master...
I20260808 16:55:52.520376 5344 heartbeater.cc:511] Master 127.8.24.62:38209 requested a full tablet report, sending...
I20260808 16:55:52.520689 4622 ts_manager.cc:194] Registered new tserver with Master: 2d4e5c451fb74bef8a43e69f5c480ac0 (127.8.24.3:37935)
I20260808 16:55:52.521034 4622 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.3@KRBTEST.COM'} at 127.8.24.3:32813
I20260808 16:55:52.527385 8288 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 16:55:52.534114 4677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:52.528601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51376 (local address 127.8.24.62:38209)
0808 16:55:52.528646 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 16:55:52.528648 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:55:52.528739 (+ 91us) server_negotiation.cc:407] Connection header received
0808 16:55:52.528790 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:55:52.528790 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:55:52.528797 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:55:52.528841 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:55:52.529332 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:52.529786 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:52.530623 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:52.530855 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:52.530910 (+ 55us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:52.532059 (+ 1149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:55:52.532060 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:55:52.532061 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:55:52.532068 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:55:52.533015 (+ 947us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:55:52.533370 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:55:52.533371 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:55:52.533372 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:55:52.533427 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:55:52.533614 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:55:52.533615 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:55:52.533615 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:55:52.533747 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:55:52.533835 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 16:55:52.533974 (+ 139us) server_negotiation.cc:299] Negotiation successful
0808 16:55:52.534031 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 16:55:52.535418 4622 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51376:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 16:55:52.536865 4622 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 16:55:52.542660 5360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:52.538753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:44387 (local address 127.0.0.1:35786)
0808 16:55:52.538990 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:52.539028 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 16:55:52.539089 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:55:52.539346 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:55:52.539349 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:55:52.539352 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:55:52.539421 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:52.539423 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:52.540745 (+ 1322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:52.540746 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:52.541524 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:52.541527 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:52.542381 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:52.542382 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:52.542460 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:52.542472 (+ 12us) client_negotiation.cc:769] Sending connection context
0808 16:55:52.542541 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 16:55:52.542591 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":50,"threads_started":1}
I20260808 16:55:52.542692 5364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:52.538933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35786 (local address 127.8.24.1:44387)
0808 16:55:52.539097 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0808 16:55:52.539101 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 16:55:52.539113 (+ 12us) server_negotiation.cc:407] Connection header received
0808 16:55:52.539183 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:55:52.539185 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:55:52.539201 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:55:52.539280 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:55:52.539550 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:52.540651 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:52.541619 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:52.542280 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:52.542384 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:52.542441 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 16:55:52.542573 (+ 132us) server_negotiation.cc:299] Negotiation successful
0808 16:55:52.542613 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":89,"threads_started":1}
I20260808 16:55:52.542901 5361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:52.538760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.2:43603 (local address 127.0.0.1:38042)
0808 16:55:52.539188 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:52.539195 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0808 16:55:52.539238 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:55:52.539495 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:55:52.539496 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:55:52.539497 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:55:52.539526 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:52.539527 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:52.540568 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:52.540569 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:52.541880 (+ 1311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:52.541882 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:52.542683 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:52.542684 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:52.542742 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:52.542746 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 16:55:52.542798 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 16:55:52.542836 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":420,"mutex_wait_us":63,"thread_start_us":55,"threads_started":1}
I20260808 16:55:52.543011 5358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:52.538754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.3:37935 (local address 127.0.0.1:45730)
0808 16:55:52.539107 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:52.539116 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 16:55:52.539154 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:55:52.539360 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:55:52.539360 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:55:52.539363 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:55:52.539429 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:52.539431 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:52.540616 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:52.540617 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:52.541879 (+ 1262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:52.541882 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:52.542785 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:52.542786 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:52.542849 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:52.542854 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 16:55:52.542900 (+ 46us) client_negotiation.cc:240] Negotiation successful
0808 16:55:52.542945 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":342,"mutex_wait_us":40,"thread_start_us":58,"threads_started":1}
I20260808 16:55:52.543172 5365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:52.538986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38042 (local address 127.8.24.2:43603)
0808 16:55:52.539147 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0808 16:55:52.539150 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 16:55:52.539242 (+ 92us) server_negotiation.cc:407] Connection header received
0808 16:55:52.539332 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:55:52.539335 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:55:52.539350 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:55:52.539418 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:55:52.539595 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:52.540474 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:52.541979 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:52.542539 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:52.542596 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:52.542649 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 16:55:52.543044 (+ 395us) server_negotiation.cc:299] Negotiation successful
0808 16:55:52.543089 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":63,"threads_started":1}
I20260808 16:55:52.543254 5363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:52.538880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45730 (local address 127.8.24.3:37935)
0808 16:55:52.539218 (+ 338us) server_negotiation.cc:206] Beginning negotiation
0808 16:55:52.539224 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 16:55:52.539232 (+ 8us) server_negotiation.cc:407] Connection header received
0808 16:55:52.539260 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:55:52.539261 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:55:52.539274 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:55:52.539360 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:55:52.539483 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:52.540537 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:52.541960 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:52.542626 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:52.542694 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:52.542747 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 16:55:52.543147 (+ 400us) server_negotiation.cc:299] Negotiation successful
0808 16:55:52.543186 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":313,"thread_start_us":77,"threads_started":1}
I20260808 16:55:52.543510 4805 tablet_service.cc:1467] Processing CreateTablet for tablet 03d398dadd6f4f99a2cebcf0cb828554 (DEFAULT_TABLE table=test-table [id=ee93a2c3e2c44f06836872dee55f3329]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:55:52.543828 4805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 03d398dadd6f4f99a2cebcf0cb828554. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:55:52.543753 5227 tablet_service.cc:1467] Processing CreateTablet for tablet 03d398dadd6f4f99a2cebcf0cb828554 (DEFAULT_TABLE table=test-table [id=ee93a2c3e2c44f06836872dee55f3329]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:55:52.544042 5227 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 03d398dadd6f4f99a2cebcf0cb828554. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:55:52.546226 5367 tablet_bootstrap.cc:492] T 03d398dadd6f4f99a2cebcf0cb828554 P 1fa6734033b542419d2801bbf29187a9: Bootstrap starting.
I20260808 16:55:52.546299 5368 tablet_bootstrap.cc:492] T 03d398dadd6f4f99a2cebcf0cb828554 P 2d4e5c451fb74bef8a43e69f5c480ac0: Bootstrap starting.
I20260808 16:55:52.546623 5014 tablet_service.cc:1467] Processing CreateTablet for tablet 03d398dadd6f4f99a2cebcf0cb828554 (DEFAULT_TABLE table=test-table [id=ee93a2c3e2c44f06836872dee55f3329]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:55:52.546939 5014 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 03d398dadd6f4f99a2cebcf0cb828554. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:55:52.546947 5368 tablet_bootstrap.cc:654] T 03d398dadd6f4f99a2cebcf0cb828554 P 2d4e5c451fb74bef8a43e69f5c480ac0: Neither blocks nor log segments found. Creating new log.
I20260808 16:55:52.547271 5367 tablet_bootstrap.cc:654] T 03d398dadd6f4f99a2cebcf0cb828554 P 1fa6734033b542419d2801bbf29187a9: Neither blocks nor log segments found. Creating new log.
I20260808 16:55:52.547451 5368 log.cc:824] T 03d398dadd6f4f99a2cebcf0cb828554 P 2d4e5c451fb74bef8a43e69f5c480ac0: Log is configured to *not* fsync() on all Append() calls
I20260808 16:55:52.547894 5367 log.cc:824] T 03d398dadd6f4f99a2cebcf0cb828554 P 1fa6734033b542419d2801bbf29187a9: Log is configured to *not* fsync() on all Append() calls
I20260808 16:55:52.548674 5367 tablet_bootstrap.cc:492] T 03d398dadd6f4f99a2cebcf0cb828554 P 1fa6734033b542419d2801bbf29187a9: No bootstrap required, opened a new log
I20260808 16:55:52.548758 5367 ts_tablet_manager.cc:1397] T 03d398dadd6f4f99a2cebcf0cb828554 P 1fa6734033b542419d2801bbf29187a9: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 16:55:52.549659 5368 tablet_bootstrap.cc:492] T 03d398dadd6f4f99a2cebcf0cb828554 P 2d4e5c451fb74bef8a43e69f5c480ac0: No bootstrap required, opened a new log
I20260808 16:55:52.549719 5368 ts_tablet_manager.cc:1397] T 03d398dadd6f4f99a2cebcf0cb828554 P 2d4e5c451fb74bef8a43e69f5c480ac0: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260808 16:55:52.549948 5375 tablet_bootstrap.cc:492] T 03d398dadd6f4f99a2cebcf0cb828554 P 3381e7c719804824b7adb2bd4fc47d3a: Bootstrap starting.
I20260808 16:55:52.550623 5375 tablet_bootstrap.cc:654] T 03d398dadd6f4f99a2cebcf0cb828554 P 3381e7c719804824b7adb2bd4fc47d3a: Neither blocks nor log segments found. Creating new log.
I20260808 16:55:52.550591 5367 raft_consensus.cc:348] T 03d398dadd6f4f99a2cebcf0cb828554 P 1fa6734033b542419d2801bbf29187a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fa6734033b542419d2801bbf29187a9" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44387 } } peers { permanent_uuid: "2d4e5c451fb74bef8a43e69f5c480ac0" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 37935 } } peers { permanent_uuid: "3381e7c719804824b7adb2bd4fc47d3a" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 43603 } }
I20260808 16:55:52.550735 5367 raft_consensus.cc:374] T 03d398dadd6f4f99a2cebcf0cb828554 P 1fa6734033b542419d2801bbf29187a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:55:52.550769 5367 raft_consensus.cc:729] T 03d398dadd6f4f99a2cebcf0cb828554 P 1fa6734033b542419d2801bbf29187a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fa6734033b542419d2801bbf29187a9, State: Initialized, Role: FOLLOWER
I20260808 16:55:52.550908 5367 consensus_queue.cc:260] T 03d398dadd6f4f99a2cebcf0cb828554 P 1fa6734033b542419d2801bbf29187a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fa6734033b542419d2801bbf29187a9" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44387 } } peers { permanent_uuid: "2d4e5c451fb74bef8a43e69f5c480ac0" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 37935 } } peers { permanent_uuid: "3381e7c719804824b7adb2bd4fc47d3a" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 43603 } }
I20260808 16:55:52.551160 4913 heartbeater.cc:503] Master 127.8.24.62:38209 was elected leader, sending a full tablet report...
I20260808 16:55:52.551196 5375 log.cc:824] T 03d398dadd6f4f99a2cebcf0cb828554 P 3381e7c719804824b7adb2bd4fc47d3a: Log is configured to *not* fsync() on all Append() calls
I20260808 16:55:52.551160 5367 ts_tablet_manager.cc:1428] T 03d398dadd6f4f99a2cebcf0cb828554 P 1fa6734033b542419d2801bbf29187a9: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 16:55:52.551602 5368 raft_consensus.cc:348] T 03d398dadd6f4f99a2cebcf0cb828554 P 2d4e5c451fb74bef8a43e69f5c480ac0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fa6734033b542419d2801bbf29187a9" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44387 } } peers { permanent_uuid: "2d4e5c451fb74bef8a43e69f5c480ac0" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 37935 } } peers { permanent_uuid: "3381e7c719804824b7adb2bd4fc47d3a" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 43603 } }
I20260808 16:55:52.551769 5368 raft_consensus.cc:374] T 03d398dadd6f4f99a2cebcf0cb828554 P 2d4e5c451fb74bef8a43e69f5c480ac0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:55:52.551816 5368 raft_consensus.cc:729] T 03d398dadd6f4f99a2cebcf0cb828554 P 2d4e5c451fb74bef8a43e69f5c480ac0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d4e5c451fb74bef8a43e69f5c480ac0, State: Initialized, Role: FOLLOWER
I20260808 16:55:52.551915 5368 consensus_queue.cc:260] T 03d398dadd6f4f99a2cebcf0cb828554 P 2d4e5c451fb74bef8a43e69f5c480ac0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fa6734033b542419d2801bbf29187a9" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44387 } } peers { permanent_uuid: "2d4e5c451fb74bef8a43e69f5c480ac0" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 37935 } } peers { permanent_uuid: "3381e7c719804824b7adb2bd4fc47d3a" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 43603 } }
I20260808 16:55:52.552170 5368 ts_tablet_manager.cc:1428] T 03d398dadd6f4f99a2cebcf0cb828554 P 2d4e5c451fb74bef8a43e69f5c480ac0: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 16:55:52.552189 5344 heartbeater.cc:503] Master 127.8.24.62:38209 was elected leader, sending a full tablet report...
I20260808 16:55:52.552779 5375 tablet_bootstrap.cc:492] T 03d398dadd6f4f99a2cebcf0cb828554 P 3381e7c719804824b7adb2bd4fc47d3a: No bootstrap required, opened a new log
I20260808 16:55:52.552840 5375 ts_tablet_manager.cc:1397] T 03d398dadd6f4f99a2cebcf0cb828554 P 3381e7c719804824b7adb2bd4fc47d3a: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260808 16:55:52.554145 5375 raft_consensus.cc:348] T 03d398dadd6f4f99a2cebcf0cb828554 P 3381e7c719804824b7adb2bd4fc47d3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fa6734033b542419d2801bbf29187a9" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44387 } } peers { permanent_uuid: "2d4e5c451fb74bef8a43e69f5c480ac0" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 37935 } } peers { permanent_uuid: "3381e7c719804824b7adb2bd4fc47d3a" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 43603 } }
I20260808 16:55:52.554286 5375 raft_consensus.cc:374] T 03d398dadd6f4f99a2cebcf0cb828554 P 3381e7c719804824b7adb2bd4fc47d3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:55:52.554318 5375 raft_consensus.cc:729] T 03d398dadd6f4f99a2cebcf0cb828554 P 3381e7c719804824b7adb2bd4fc47d3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3381e7c719804824b7adb2bd4fc47d3a, State: Initialized, Role: FOLLOWER
I20260808 16:55:52.554445 5375 consensus_queue.cc:260] T 03d398dadd6f4f99a2cebcf0cb828554 P 3381e7c719804824b7adb2bd4fc47d3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fa6734033b542419d2801bbf29187a9" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44387 } } peers { permanent_uuid: "2d4e5c451fb74bef8a43e69f5c480ac0" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 37935 } } peers { permanent_uuid: "3381e7c719804824b7adb2bd4fc47d3a" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 43603 } }
I20260808 16:55:52.554663 5375 ts_tablet_manager.cc:1428] T 03d398dadd6f4f99a2cebcf0cb828554 P 3381e7c719804824b7adb2bd4fc47d3a: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 16:55:52.554701 5114 heartbeater.cc:503] Master 127.8.24.62:38209 was elected leader, sending a full tablet report...
I20260808 16:55:52.602705 5379 raft_consensus.cc:482] T 03d398dadd6f4f99a2cebcf0cb828554 P 3381e7c719804824b7adb2bd4fc47d3a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 16:55:52.602815 5379 raft_consensus.cc:504] T 03d398dadd6f4f99a2cebcf0cb828554 P 3381e7c719804824b7adb2bd4fc47d3a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fa6734033b542419d2801bbf29187a9" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44387 } } peers { permanent_uuid: "2d4e5c451fb74bef8a43e69f5c480ac0" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 37935 } } peers { permanent_uuid: "3381e7c719804824b7adb2bd4fc47d3a" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 43603 } }
I20260808 16:55:52.603135 5379 leader_election.cc:288] T 03d398dadd6f4f99a2cebcf0cb828554 P 3381e7c719804824b7adb2bd4fc47d3a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1fa6734033b542419d2801bbf29187a9 (127.8.24.1:44387), 2d4e5c451fb74bef8a43e69f5c480ac0 (127.8.24.3:37935)
W20260808 16:55:52.605917 5115 tablet.cc:2369] T 03d398dadd6f4f99a2cebcf0cb828554 P 3381e7c719804824b7adb2bd4fc47d3a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 16:55:52.607020 5364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:52.603364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:34209 (local address 127.8.24.1:44387)
0808 16:55:52.603471 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0808 16:55:52.603473 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:55:52.603652 (+ 179us) server_negotiation.cc:407] Connection header received
0808 16:55:52.603793 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:55:52.603795 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:55:52.603822 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:55:52.603852 (+ 30us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:55:52.604037 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:52.604694 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:52.606103 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:52.606678 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:52.606722 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:52.606732 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 16:55:52.606921 (+ 189us) server_negotiation.cc:299] Negotiation successful
0808 16:55:52.606959 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260808 16:55:52.607085 5116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:52.603295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.3:37935 (local address 127.8.24.2:49071)
0808 16:55:52.603340 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:52.603351 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 16:55:52.603392 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:55:52.603652 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:55:52.603653 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:55:52.603654 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:55:52.603718 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:52.603720 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:52.604591 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:52.604592 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:52.605743 (+ 1151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:52.605746 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:52.606841 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:52.606842 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:52.606926 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:52.606930 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 16:55:52.606978 (+ 48us) client_negotiation.cc:240] Negotiation successful
0808 16:55:52.607016 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260808 16:55:52.607371 5363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:52.603414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:49071 (local address 127.8.24.3:37935)
0808 16:55:52.603526 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0808 16:55:52.603528 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:55:52.603537 (+ 9us) server_negotiation.cc:407] Connection header received
0808 16:55:52.603562 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:55:52.603563 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:55:52.603574 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:55:52.603631 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:55:52.603775 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:52.604501 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:52.606136 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:52.606721 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:52.606796 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:52.606807 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 16:55:52.607269 (+ 462us) server_negotiation.cc:299] Negotiation successful
0808 16:55:52.607312 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260808 16:55:52.607626 5380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:52.603295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:44387 (local address 127.8.24.2:34209)
0808 16:55:52.603577 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:52.603586 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 16:55:52.603676 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:55:52.603909 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:55:52.603910 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:55:52.603911 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:55:52.603968 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:52.603969 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:52.604793 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:52.604795 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:52.605923 (+ 1128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:52.605927 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:52.606787 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:52.606788 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:52.606854 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:52.606857 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 16:55:52.607332 (+ 475us) client_negotiation.cc:240] Negotiation successful
0808 16:55:52.607371 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"mutex_wait_us":140,"thread_start_us":86,"threads_started":1}
I20260808 16:55:52.612946 5256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03d398dadd6f4f99a2cebcf0cb828554" candidate_uuid: "3381e7c719804824b7adb2bd4fc47d3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d4e5c451fb74bef8a43e69f5c480ac0" is_pre_election: true
I20260808 16:55:52.613092 5256 raft_consensus.cc:2454] T 03d398dadd6f4f99a2cebcf0cb828554 P 2d4e5c451fb74bef8a43e69f5c480ac0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3381e7c719804824b7adb2bd4fc47d3a in term 0.
I20260808 16:55:52.613349 4949 leader_election.cc:302] T 03d398dadd6f4f99a2cebcf0cb828554 P 3381e7c719804824b7adb2bd4fc47d3a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d4e5c451fb74bef8a43e69f5c480ac0, 3381e7c719804824b7adb2bd4fc47d3a; no voters:
I20260808 16:55:52.613776 5379 raft_consensus.cc:2781] T 03d398dadd6f4f99a2cebcf0cb828554 P 3381e7c719804824b7adb2bd4fc47d3a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 16:55:52.613848 5379 raft_consensus.cc:482] T 03d398dadd6f4f99a2cebcf0cb828554 P 3381e7c719804824b7adb2bd4fc47d3a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 16:55:52.613873 5379 raft_consensus.cc:3037] T 03d398dadd6f4f99a2cebcf0cb828554 P 3381e7c719804824b7adb2bd4fc47d3a [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:55:52.614786 5379 raft_consensus.cc:504] T 03d398dadd6f4f99a2cebcf0cb828554 P 3381e7c719804824b7adb2bd4fc47d3a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fa6734033b542419d2801bbf29187a9" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44387 } } peers { permanent_uuid: "2d4e5c451fb74bef8a43e69f5c480ac0" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 37935 } } peers { permanent_uuid: "3381e7c719804824b7adb2bd4fc47d3a" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 43603 } }
I20260808 16:55:52.614926 5379 leader_election.cc:288] T 03d398dadd6f4f99a2cebcf0cb828554 P 3381e7c719804824b7adb2bd4fc47d3a [CANDIDATE]: Term 1 election: Requested vote from peers 1fa6734033b542419d2801bbf29187a9 (127.8.24.1:44387), 2d4e5c451fb74bef8a43e69f5c480ac0 (127.8.24.3:37935)
I20260808 16:55:52.614874 4831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03d398dadd6f4f99a2cebcf0cb828554" candidate_uuid: "3381e7c719804824b7adb2bd4fc47d3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fa6734033b542419d2801bbf29187a9" is_pre_election: true
I20260808 16:55:52.615018 4831 raft_consensus.cc:2454] T 03d398dadd6f4f99a2cebcf0cb828554 P 1fa6734033b542419d2801bbf29187a9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3381e7c719804824b7adb2bd4fc47d3a in term 0.
I20260808 16:55:52.615095 5256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03d398dadd6f4f99a2cebcf0cb828554" candidate_uuid: "3381e7c719804824b7adb2bd4fc47d3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d4e5c451fb74bef8a43e69f5c480ac0"
I20260808 16:55:52.615078 4829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03d398dadd6f4f99a2cebcf0cb828554" candidate_uuid: "3381e7c719804824b7adb2bd4fc47d3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fa6734033b542419d2801bbf29187a9"
I20260808 16:55:52.615164 5256 raft_consensus.cc:3037] T 03d398dadd6f4f99a2cebcf0cb828554 P 2d4e5c451fb74bef8a43e69f5c480ac0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:55:52.615164 4829 raft_consensus.cc:3037] T 03d398dadd6f4f99a2cebcf0cb828554 P 1fa6734033b542419d2801bbf29187a9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:55:52.615986 5256 raft_consensus.cc:2454] T 03d398dadd6f4f99a2cebcf0cb828554 P 2d4e5c451fb74bef8a43e69f5c480ac0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3381e7c719804824b7adb2bd4fc47d3a in term 1.
I20260808 16:55:52.615986 4829 raft_consensus.cc:2454] T 03d398dadd6f4f99a2cebcf0cb828554 P 1fa6734033b542419d2801bbf29187a9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3381e7c719804824b7adb2bd4fc47d3a in term 1.
I20260808 16:55:52.616147 4949 leader_election.cc:302] T 03d398dadd6f4f99a2cebcf0cb828554 P 3381e7c719804824b7adb2bd4fc47d3a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d4e5c451fb74bef8a43e69f5c480ac0, 3381e7c719804824b7adb2bd4fc47d3a; no voters:
I20260808 16:55:52.616281 5379 raft_consensus.cc:2781] T 03d398dadd6f4f99a2cebcf0cb828554 P 3381e7c719804824b7adb2bd4fc47d3a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:55:52.616413 5379 raft_consensus.cc:686] T 03d398dadd6f4f99a2cebcf0cb828554 P 3381e7c719804824b7adb2bd4fc47d3a [term 1 LEADER]: Becoming Leader. State: Replica: 3381e7c719804824b7adb2bd4fc47d3a, State: Running, Role: LEADER
I20260808 16:55:52.616515 5379 consensus_queue.cc:237] T 03d398dadd6f4f99a2cebcf0cb828554 P 3381e7c719804824b7adb2bd4fc47d3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fa6734033b542419d2801bbf29187a9" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44387 } } peers { permanent_uuid: "2d4e5c451fb74bef8a43e69f5c480ac0" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 37935 } } peers { permanent_uuid: "3381e7c719804824b7adb2bd4fc47d3a" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 43603 } }
I20260808 16:55:52.617287 4619 catalog_manager.cc:5495] T 03d398dadd6f4f99a2cebcf0cb828554 P 3381e7c719804824b7adb2bd4fc47d3a reported cstate change: term changed from 0 to 1, leader changed from <none> to 3381e7c719804824b7adb2bd4fc47d3a (127.8.24.2). New cstate: current_term: 1 leader_uuid: "3381e7c719804824b7adb2bd4fc47d3a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fa6734033b542419d2801bbf29187a9" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d4e5c451fb74bef8a43e69f5c480ac0" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 37935 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3381e7c719804824b7adb2bd4fc47d3a" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 43603 } health_report { overall_health: HEALTHY } } }
I20260808 16:55:52.636799 5365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:52.634686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38046 (local address 127.8.24.2:43603)
0808 16:55:52.634744 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 16:55:52.634746 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:55:52.634753 (+ 7us) server_negotiation.cc:407] Connection header received
0808 16:55:52.634779 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:55:52.634780 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:55:52.634795 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:55:52.634853 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 16:55:52.634963 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:52.635396 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:52.636105 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:52.636299 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:52.636352 (+ 53us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:52.636503 (+ 151us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 16:55:52.636584 (+ 81us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 16:55:52.636663 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 16:55:52.636728 (+ 65us) server_negotiation.cc:299] Negotiation successful
0808 16:55:52.636736 (+ 8us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 16:55:52.638202 5256 raft_consensus.cc:1261] T 03d398dadd6f4f99a2cebcf0cb828554 P 2d4e5c451fb74bef8a43e69f5c480ac0 [term 1 FOLLOWER]: Refusing update from remote peer 3381e7c719804824b7adb2bd4fc47d3a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:55:52.638219 4829 raft_consensus.cc:1261] T 03d398dadd6f4f99a2cebcf0cb828554 P 1fa6734033b542419d2801bbf29187a9 [term 1 FOLLOWER]: Refusing update from remote peer 3381e7c719804824b7adb2bd4fc47d3a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:55:52.638537 5382 consensus_queue.cc:1035] T 03d398dadd6f4f99a2cebcf0cb828554 P 3381e7c719804824b7adb2bd4fc47d3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d4e5c451fb74bef8a43e69f5c480ac0" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 37935 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:55:52.638670 5379 consensus_queue.cc:1035] T 03d398dadd6f4f99a2cebcf0cb828554 P 3381e7c719804824b7adb2bd4fc47d3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fa6734033b542419d2801bbf29187a9" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:55:52.641060 5387 mvcc.cc:205] Tried to move back new op lower bound from 7316308593203974144 to 7316308593117749248. Current Snapshot: MvccSnapshot[applied={T|T < 7316308593203974144}]
I20260808 16:55:52.643157 4622 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51376:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 16:55:52.643253 4622 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51376:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 16:55:52.643913 4622 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c390815c100f47a19114e87e07c509fd: Sending DeleteTablet for 3 replicas of tablet 03d398dadd6f4f99a2cebcf0cb828554
I20260808 16:55:52.644219 4805 tablet_service.cc:1514] Processing DeleteTablet for tablet 03d398dadd6f4f99a2cebcf0cb828554 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:55:52 UTC) from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:35786
I20260808 16:55:52.644222 5014 tablet_service.cc:1514] Processing DeleteTablet for tablet 03d398dadd6f4f99a2cebcf0cb828554 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:55:52 UTC) from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:38042
I20260808 16:55:52.644470 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 4709
I20260808 16:55:52.644562 5227 tablet_service.cc:1514] Processing DeleteTablet for tablet 03d398dadd6f4f99a2cebcf0cb828554 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:55:52 UTC) from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:45730
I20260808 16:55:52.654279 5404 tablet_replica.cc:330] T 03d398dadd6f4f99a2cebcf0cb828554 P 3381e7c719804824b7adb2bd4fc47d3a: stopping tablet replica
I20260808 16:55:52.654407 5404 raft_consensus.cc:2229] T 03d398dadd6f4f99a2cebcf0cb828554 P 3381e7c719804824b7adb2bd4fc47d3a [term 1 LEADER]: Raft consensus shutting down.
I20260808 16:55:52.654503 5404 raft_consensus.cc:2258] T 03d398dadd6f4f99a2cebcf0cb828554 P 3381e7c719804824b7adb2bd4fc47d3a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:55:52.654863 5404 ts_tablet_manager.cc:1905] T 03d398dadd6f4f99a2cebcf0cb828554 P 3381e7c719804824b7adb2bd4fc47d3a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:55:52.656185 5404 ts_tablet_manager.cc:1918] T 03d398dadd6f4f99a2cebcf0cb828554 P 3381e7c719804824b7adb2bd4fc47d3a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 16:55:52.656252 5404 log.cc:1196] T 03d398dadd6f4f99a2cebcf0cb828554 P 3381e7c719804824b7adb2bd4fc47d3a: Deleting WAL directory at /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786207919384338-8288-0/minicluster-data/ts-1/wal/wals/03d398dadd6f4f99a2cebcf0cb828554
I20260808 16:55:52.656450 5404 ts_tablet_manager.cc:1939] T 03d398dadd6f4f99a2cebcf0cb828554 P 3381e7c719804824b7adb2bd4fc47d3a: Deleting consensus metadata
I20260808 16:55:52.656814 4598 catalog_manager.cc:4841] TS 3381e7c719804824b7adb2bd4fc47d3a (127.8.24.2:43603): tablet 03d398dadd6f4f99a2cebcf0cb828554 (table test-table [id=ee93a2c3e2c44f06836872dee55f3329]) successfully deleted
W20260808 16:55:52.661651 4596 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.24.1:44387 (error 108)
I20260808 16:55:52.661742 5410 tablet_replica.cc:330] T 03d398dadd6f4f99a2cebcf0cb828554 P 2d4e5c451fb74bef8a43e69f5c480ac0: stopping tablet replica
I20260808 16:55:52.661865 5410 raft_consensus.cc:2229] T 03d398dadd6f4f99a2cebcf0cb828554 P 2d4e5c451fb74bef8a43e69f5c480ac0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 16:55:52.661899 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 4920
I20260808 16:55:52.661981 5410 raft_consensus.cc:2258] T 03d398dadd6f4f99a2cebcf0cb828554 P 2d4e5c451fb74bef8a43e69f5c480ac0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:55:52.662096 5358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:52.661994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:44387 (local address 127.0.0.1:35800)
0808 16:55:52.662039 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:52.662069 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.1:44387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260808 16:55:52.664582 5410 ts_tablet_manager.cc:1905] T 03d398dadd6f4f99a2cebcf0cb828554 P 2d4e5c451fb74bef8a43e69f5c480ac0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:55:52.666774 5410 ts_tablet_manager.cc:1918] T 03d398dadd6f4f99a2cebcf0cb828554 P 2d4e5c451fb74bef8a43e69f5c480ac0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 16:55:52.666844 5410 log.cc:1196] T 03d398dadd6f4f99a2cebcf0cb828554 P 2d4e5c451fb74bef8a43e69f5c480ac0: Deleting WAL directory at /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786207919384338-8288-0/minicluster-data/ts-2/wal/wals/03d398dadd6f4f99a2cebcf0cb828554
I20260808 16:55:52.667053 5410 ts_tablet_manager.cc:1939] T 03d398dadd6f4f99a2cebcf0cb828554 P 2d4e5c451fb74bef8a43e69f5c480ac0: Deleting consensus metadata
W20260808 16:55:52.667047 4596 catalog_manager.cc:4584] TS 1fa6734033b542419d2801bbf29187a9 (127.8.24.1:44387): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 03d398dadd6f4f99a2cebcf0cb828554: Network error: Client connection negotiation failed: client connection to 127.8.24.1:44387: connect: Connection refused (error 111)
I20260808 16:55:52.667415 4599 catalog_manager.cc:4841] TS 2d4e5c451fb74bef8a43e69f5c480ac0 (127.8.24.3:37935): tablet 03d398dadd6f4f99a2cebcf0cb828554 (table test-table [id=ee93a2c3e2c44f06836872dee55f3329]) successfully deleted
I20260808 16:55:52.671237 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 5119
I20260808 16:55:52.680083 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 4561
2026-08-08T16:55:52Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2020 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-08T16:55:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T16:55:52Z Disabled control of system clock
I20260808 16:55:52.709610 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786207919384338-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786207919384338-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786207919384338-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:37219
--webserver_interface=127.8.24.62
--webserver_port=0
--builtin_ntp_servers=127.8.24.20:43899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:37219
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 16:55:52.787873 5418 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:55:52.788062 5418 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:55:52.788081 5418 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:55:52.789139 5418 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 16:55:52.789171 5418 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:55:52.789182 5418 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:55:52.789193 5418 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 16:55:52.789209 5418 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 16:55:52.790508 5418 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:43899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786207919384338-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786207919384338-8288-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:37219
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:37219
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.24.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5418
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786207919384338-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:55:52.790692 5418 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:55:52.790920 5418 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:55:52.795086 5427 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:55:52.795077 5428 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:55:52.795077 5430 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:55:52.795475 5418 server_base.cc:1034] running on GCE node
I20260808 16:55:52.795668 5418 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:55:52.795908 5418 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:55:52.797043 5418 hybrid_clock.cc:648] HybridClock initialized: now 1786208152797029 us; error 31 us; skew 500 ppm
I20260808 16:55:52.798100 5418 webserver.cc:460] Webserver started at http://127.8.24.62:43991/ using document root <none> and password file <none>
I20260808 16:55:52.798310 5418 fs_manager.cc:359] Metadata directory not provided
I20260808 16:55:52.798388 5418 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:55:52.798512 5418 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:55:52.799446 5418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786207919384338-8288-0/minicluster-data/master-0/data/instance:
uuid: "dd603eb005ae4aa6bd01c11281c87a0f"
format_stamp: "Formatted at 2026-08-08 16:55:52 on dist-test-slave-6zbq"
server_key: "caf9b1b0eb479da673f48fcebbfcc0ed"
server_key_iv: "55dc6d066f073810c832be1b98df8ca7"
server_key_version: "encryptionkey@0"
I20260808 16:55:52.799754 5418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786207919384338-8288-0/minicluster-data/master-0/wal/instance:
uuid: "dd603eb005ae4aa6bd01c11281c87a0f"
format_stamp: "Formatted at 2026-08-08 16:55:52 on dist-test-slave-6zbq"
server_key: "caf9b1b0eb479da673f48fcebbfcc0ed"
server_key_iv: "55dc6d066f073810c832be1b98df8ca7"
server_key_version: "encryptionkey@0"
I20260808 16:55:52.800863 5418 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 16:55:52.801465 5442 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:55:52.801609 5418 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 16:55:52.801688 5418 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786207919384338-8288-0/minicluster-data/master-0/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786207919384338-8288-0/minicluster-data/master-0/wal
uuid: "dd603eb005ae4aa6bd01c11281c87a0f"
format_stamp: "Formatted at 2026-08-08 16:55:52 on dist-test-slave-6zbq"
server_key: "caf9b1b0eb479da673f48fcebbfcc0ed"
server_key_iv: "55dc6d066f073810c832be1b98df8ca7"
server_key_version: "encryptionkey@0"
I20260808 16:55:52.801745 5418 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786207919384338-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786207919384338-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786207919384338-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:55:52.827471 5418 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:55:52.827714 5418 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:55:52.827842 5418 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:55:52.831305 5418 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.62:37219
I20260808 16:55:52.831336 5526 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.62:37219 every 8 connection(s)
I20260808 16:55:52.831609 5418 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
I20260808 16:55:52.832461 5527 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:55:52.833500 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 5418
I20260808 16:55:52.833583 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786207919384338-8288-0/minicluster-data/master-0/wal/instance
I20260808 16:55:52.833755 8288 external_mini_cluster.cc:1428] Setting key e0d39b9ac16db78c59dea5e491d6eac7
I20260808 16:55:52.834899 5527 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd603eb005ae4aa6bd01c11281c87a0f: Bootstrap starting.
I20260808 16:55:52.835564 5527 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dd603eb005ae4aa6bd01c11281c87a0f: Neither blocks nor log segments found. Creating new log.
I20260808 16:55:52.835882 5527 log.cc:824] T 00000000000000000000000000000000 P dd603eb005ae4aa6bd01c11281c87a0f: Log is configured to *not* fsync() on all Append() calls
I20260808 16:55:52.836553 5527 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd603eb005ae4aa6bd01c11281c87a0f: No bootstrap required, opened a new log
I20260808 16:55:52.838577 5527 raft_consensus.cc:348] T 00000000000000000000000000000000 P dd603eb005ae4aa6bd01c11281c87a0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd603eb005ae4aa6bd01c11281c87a0f" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 37219 } }
I20260808 16:55:52.838748 5527 raft_consensus.cc:374] T 00000000000000000000000000000000 P dd603eb005ae4aa6bd01c11281c87a0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:55:52.838806 5527 raft_consensus.cc:729] T 00000000000000000000000000000000 P dd603eb005ae4aa6bd01c11281c87a0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd603eb005ae4aa6bd01c11281c87a0f, State: Initialized, Role: FOLLOWER
I20260808 16:55:52.838933 5527 consensus_queue.cc:260] T 00000000000000000000000000000000 P dd603eb005ae4aa6bd01c11281c87a0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd603eb005ae4aa6bd01c11281c87a0f" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 37219 } }
I20260808 16:55:52.839006 5527 raft_consensus.cc:388] T 00000000000000000000000000000000 P dd603eb005ae4aa6bd01c11281c87a0f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 16:55:52.839044 5527 raft_consensus.cc:482] T 00000000000000000000000000000000 P dd603eb005ae4aa6bd01c11281c87a0f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 16:55:52.839094 5527 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dd603eb005ae4aa6bd01c11281c87a0f [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:55:52.839291 5533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:52.834186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60020 (local address 127.8.24.62:37219)
0808 16:55:52.834396 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0808 16:55:52.834405 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 16:55:52.834445 (+ 40us) server_negotiation.cc:407] Connection header received
0808 16:55:52.834898 (+ 453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:55:52.834906 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:55:52.834983 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:55:52.835126 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:55:52.835295 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:52.836509 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:52.837278 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:52.837871 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:52.837999 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:52.838491 (+ 492us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:55:52.838495 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:55:52.838495 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 16:55:52.838677 (+ 182us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:55:52.838723 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 16:55:52.838738 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:55:52.838807 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 16:55:52.838967 (+ 160us) server_negotiation.cc:299] Negotiation successful
0808 16:55:52.839113 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":72,"threads_started":1}
I20260808 16:55:52.839655 5527 raft_consensus.cc:504] T 00000000000000000000000000000000 P dd603eb005ae4aa6bd01c11281c87a0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd603eb005ae4aa6bd01c11281c87a0f" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 37219 } }
I20260808 16:55:52.839766 5527 leader_election.cc:302] T 00000000000000000000000000000000 P dd603eb005ae4aa6bd01c11281c87a0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dd603eb005ae4aa6bd01c11281c87a0f; no voters:
I20260808 16:55:52.839910 5527 leader_election.cc:288] T 00000000000000000000000000000000 P dd603eb005ae4aa6bd01c11281c87a0f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 16:55:52.839934 5536 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dd603eb005ae4aa6bd01c11281c87a0f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:55:52.840032 5536 raft_consensus.cc:686] T 00000000000000000000000000000000 P dd603eb005ae4aa6bd01c11281c87a0f [term 1 LEADER]: Becoming Leader. State: Replica: dd603eb005ae4aa6bd01c11281c87a0f, State: Running, Role: LEADER
I20260808 16:55:52.840112 5536 consensus_queue.cc:237] T 00000000000000000000000000000000 P dd603eb005ae4aa6bd01c11281c87a0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd603eb005ae4aa6bd01c11281c87a0f" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 37219 } }
I20260808 16:55:52.840144 5527 sys_catalog.cc:564] T 00000000000000000000000000000000 P dd603eb005ae4aa6bd01c11281c87a0f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 16:55:52.840466 5540 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd603eb005ae4aa6bd01c11281c87a0f [sys.catalog]: SysCatalogTable state changed. Reason: New leader dd603eb005ae4aa6bd01c11281c87a0f. Latest consensus state: current_term: 1 leader_uuid: "dd603eb005ae4aa6bd01c11281c87a0f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd603eb005ae4aa6bd01c11281c87a0f" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 37219 } } }
I20260808 16:55:52.840554 5540 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd603eb005ae4aa6bd01c11281c87a0f [sys.catalog]: This master's current role is: LEADER
I20260808 16:55:52.840730 5537 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd603eb005ae4aa6bd01c11281c87a0f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dd603eb005ae4aa6bd01c11281c87a0f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd603eb005ae4aa6bd01c11281c87a0f" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 37219 } } }
I20260808 16:55:52.840780 5544 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 16:55:52.840808 5537 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd603eb005ae4aa6bd01c11281c87a0f [sys.catalog]: This master's current role is: LEADER
I20260808 16:55:52.841287 5544 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 16:55:52.842692 5544 catalog_manager.cc:1347] Generated new cluster ID: fd374ab3167843a597e27b3763af4061
I20260808 16:55:52.842742 5544 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 16:55:52.849977 5544 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 16:55:52.850495 5544 catalog_manager.cc:1504] Loading token signing keys...
I20260808 16:55:52.855748 5544 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dd603eb005ae4aa6bd01c11281c87a0f: Generated new TSK 0
I20260808 16:55:52.855893 5544 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 16:55:52.857316 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786207919384338-8288-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786207919384338-8288-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786207919384338-8288-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786207919384338-8288-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.1:0
--local_ip_for_outbound_sockets=127.8.24.1
--webserver_interface=127.8.24.1
--webserver_port=0
--tserver_master_addrs=127.8.24.62:37219
--builtin_ntp_servers=127.8.24.20:43899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 16:55:52.945717 5563 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:55:52.945889 5563 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:55:52.945909 5563 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:55:52.947007 5563 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:55:52.947042 5563 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:55:52.947065 5563 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.1
I20260808 16:55:52.948352 5563 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:43899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786207919384338-8288-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786207919384338-8288-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786207919384338-8288-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.24.1
--webserver_port=0
--tserver_master_addrs=127.8.24.62:37219
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5563
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.1
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786207919384338-8288-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:55:52.948559 5563 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:55:52.948833 5563 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:55:52.953306 5568 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:55:52.953306 5570 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:55:52.953305 5572 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:55:52.953656 5563 server_base.cc:1034] running on GCE node
I20260808 16:55:52.953841 5563 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:55:52.954039 5563 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:55:52.955169 5563 hybrid_clock.cc:648] HybridClock initialized: now 1786208152955157 us; error 32 us; skew 500 ppm
I20260808 16:55:52.956240 5563 webserver.cc:460] Webserver started at http://127.8.24.1:36161/ using document root <none> and password file <none>
I20260808 16:55:52.956450 5563 fs_manager.cc:359] Metadata directory not provided
I20260808 16:55:52.956497 5563 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:55:52.956614 5563 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:55:52.957536 5563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786207919384338-8288-0/minicluster-data/ts-0/data/instance:
uuid: "0140c1c94154476eb82c96d55cc05e6b"
format_stamp: "Formatted at 2026-08-08 16:55:52 on dist-test-slave-6zbq"
server_key: "c2a5f52133773a88ca12aa522be7539a"
server_key_iv: "6b83751f5a1e4946c0e8d54d41a6e67b"
server_key_version: "encryptionkey@0"
I20260808 16:55:52.957854 5563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786207919384338-8288-0/minicluster-data/ts-0/wal/instance:
uuid: "0140c1c94154476eb82c96d55cc05e6b"
format_stamp: "Formatted at 2026-08-08 16:55:52 on dist-test-slave-6zbq"
server_key: "c2a5f52133773a88ca12aa522be7539a"
server_key_iv: "6b83751f5a1e4946c0e8d54d41a6e67b"
server_key_version: "encryptionkey@0"
I20260808 16:55:52.959090 5563 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 16:55:52.959772 5580 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:55:52.959923 5563 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 16:55:52.960001 5563 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786207919384338-8288-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786207919384338-8288-0/minicluster-data/ts-0/wal
uuid: "0140c1c94154476eb82c96d55cc05e6b"
format_stamp: "Formatted at 2026-08-08 16:55:52 on dist-test-slave-6zbq"
server_key: "c2a5f52133773a88ca12aa522be7539a"
server_key_iv: "6b83751f5a1e4946c0e8d54d41a6e67b"
server_key_version: "encryptionkey@0"
I20260808 16:55:52.960063 5563 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786207919384338-8288-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786207919384338-8288-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786207919384338-8288-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:55:52.970150 5563 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:55:52.970408 5563 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:55:52.970520 5563 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:55:52.970701 5563 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:55:52.970963 5563 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:55:52.970992 5563 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:55:52.971019 5563 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:55:52.971041 5563 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:55:52.976289 5563 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.1:40445
I20260808 16:55:52.976372 5744 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.1:40445 every 8 connection(s)
I20260808 16:55:52.976589 5563 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786207919384338-8288-0/minicluster-data/ts-0/data/info.pb
I20260808 16:55:52.980945 5533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:52.977208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:38757 (local address 127.8.24.62:37219)
0808 16:55:52.977255 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 16:55:52.977258 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 16:55:52.977486 (+ 228us) server_negotiation.cc:407] Connection header received
0808 16:55:52.977917 (+ 431us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:55:52.977917 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:55:52.977927 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:55:52.977974 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:55:52.978575 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:52.979046 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:52.979913 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:52.980084 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:52.980125 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:52.980771 (+ 646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:55:52.980772 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:55:52.980772 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 16:55:52.980775 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:55:52.980790 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 16:55:52.980793 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:55:52.980820 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0808 16:55:52.980877 (+ 57us) server_negotiation.cc:299] Negotiation successful
0808 16:55:52.980891 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 16:55:52.981127 5748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:52.977195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:37219 (local address 127.8.24.1:38757)
0808 16:55:52.977386 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:52.977416 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0808 16:55:52.977845 (+ 429us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:55:52.978021 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:55:52.978024 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:55:52.978032 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:55:52.978514 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:52.978520 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:52.979104 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:52.979104 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:52.979857 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:52.979858 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:52.980133 (+ 275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:52.980134 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:52.980229 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:52.980656 (+ 427us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 16:55:52.980657 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:55:52.980691 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 16:55:52.980713 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:55:52.980830 (+ 117us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:55:52.980832 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:55:52.980834 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 16:55:52.980875 (+ 41us) client_negotiation.cc:240] Negotiation successful
0808 16:55:52.980942 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":101,"threads_started":1}
I20260808 16:55:52.981568 5745 heartbeater.cc:347] Connected to a master server at 127.8.24.62:37219
I20260808 16:55:52.981644 5745 heartbeater.cc:464] Registering TS with master...
I20260808 16:55:52.981784 5745 heartbeater.cc:511] Master 127.8.24.62:37219 requested a full tablet report, sending...
I20260808 16:55:52.982210 5469 ts_manager.cc:194] Registered new tserver with Master: 0140c1c94154476eb82c96d55cc05e6b (127.8.24.1:40445)
I20260808 16:55:52.982837 5469 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.24.1:38757
I20260808 16:55:52.982908 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 5563
I20260808 16:55:52.983004 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786207919384338-8288-0/minicluster-data/ts-0/wal/instance
I20260808 16:55:52.983211 8288 external_mini_cluster.cc:1428] Setting key e88fdf0b195d10a2e038807801cd79b0
I20260808 16:55:52.984063 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786207919384338-8288-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786207919384338-8288-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786207919384338-8288-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786207919384338-8288-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.2:0
--local_ip_for_outbound_sockets=127.8.24.2
--webserver_interface=127.8.24.2
--webserver_port=0
--tserver_master_addrs=127.8.24.62:37219
--builtin_ntp_servers=127.8.24.20:43899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 16:55:53.062041 5749 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:55:53.062247 5749 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:55:53.062284 5749 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:55:53.063423 5749 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:55:53.063462 5749 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:55:53.063496 5749 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.2
I20260808 16:55:53.064807 5749 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:43899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786207919384338-8288-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786207919384338-8288-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786207919384338-8288-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.24.2
--webserver_port=0
--tserver_master_addrs=127.8.24.62:37219
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5749
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.2
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786207919384338-8288-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:55:53.065023 5749 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:55:53.065275 5749 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:55:53.069716 5758 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:55:53.069713 5760 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:55:53.069906 5749 server_base.cc:1034] running on GCE node
W20260808 16:55:53.070009 5756 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:55:53.070192 5749 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:55:53.070426 5749 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:55:53.071557 5749 hybrid_clock.cc:648] HybridClock initialized: now 1786208153071539 us; error 37 us; skew 500 ppm
I20260808 16:55:53.072679 5749 webserver.cc:460] Webserver started at http://127.8.24.2:44075/ using document root <none> and password file <none>
I20260808 16:55:53.072889 5749 fs_manager.cc:359] Metadata directory not provided
I20260808 16:55:53.072932 5749 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:55:53.073050 5749 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:55:53.073980 5749 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786207919384338-8288-0/minicluster-data/ts-1/data/instance:
uuid: "b62ec01c05734fbc98e9d87236a212bf"
format_stamp: "Formatted at 2026-08-08 16:55:53 on dist-test-slave-6zbq"
server_key: "dc0902b15a16983fcb9d95b981bcc91d"
server_key_iv: "5544cb7fbe31f9d0529f21a9dd7f1dcd"
server_key_version: "encryptionkey@0"
I20260808 16:55:53.074276 5749 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786207919384338-8288-0/minicluster-data/ts-1/wal/instance:
uuid: "b62ec01c05734fbc98e9d87236a212bf"
format_stamp: "Formatted at 2026-08-08 16:55:53 on dist-test-slave-6zbq"
server_key: "dc0902b15a16983fcb9d95b981bcc91d"
server_key_iv: "5544cb7fbe31f9d0529f21a9dd7f1dcd"
server_key_version: "encryptionkey@0"
I20260808 16:55:53.075412 5749 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 16:55:53.076016 5770 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:55:53.076148 5749 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 16:55:53.076221 5749 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786207919384338-8288-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786207919384338-8288-0/minicluster-data/ts-1/wal
uuid: "b62ec01c05734fbc98e9d87236a212bf"
format_stamp: "Formatted at 2026-08-08 16:55:53 on dist-test-slave-6zbq"
server_key: "dc0902b15a16983fcb9d95b981bcc91d"
server_key_iv: "5544cb7fbe31f9d0529f21a9dd7f1dcd"
server_key_version: "encryptionkey@0"
I20260808 16:55:53.076282 5749 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786207919384338-8288-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786207919384338-8288-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786207919384338-8288-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:55:53.089291 5749 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:55:53.089519 5749 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:55:53.089653 5749 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:55:53.089836 5749 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:55:53.090140 5749 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:55:53.090181 5749 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:55:53.090207 5749 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:55:53.090230 5749 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:55:53.095475 5749 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.2:34469
I20260808 16:55:53.095538 5936 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.2:34469 every 8 connection(s)
I20260808 16:55:53.095788 5749 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786207919384338-8288-0/minicluster-data/ts-1/data/info.pb
I20260808 16:55:53.098834 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 5749
I20260808 16:55:53.098912 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786207919384338-8288-0/minicluster-data/ts-1/wal/instance
I20260808 16:55:53.099086 8288 external_mini_cluster.cc:1428] Setting key f623289b703cb215e1b7bf93ab96e337
I20260808 16:55:53.100033 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786207919384338-8288-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786207919384338-8288-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786207919384338-8288-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786207919384338-8288-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.3:0
--local_ip_for_outbound_sockets=127.8.24.3
--webserver_interface=127.8.24.3
--webserver_port=0
--tserver_master_addrs=127.8.24.62:37219
--builtin_ntp_servers=127.8.24.20:43899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 16:55:53.100378 5533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:53.096374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:47195 (local address 127.8.24.62:37219)
0808 16:55:53.096424 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 16:55:53.096426 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:55:53.096638 (+ 212us) server_negotiation.cc:407] Connection header received
0808 16:55:53.097024 (+ 386us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:55:53.097025 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:55:53.097035 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:55:53.097077 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:55:53.097621 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:53.098091 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:53.099089 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:53.099319 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:53.099380 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:53.100129 (+ 749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:55:53.100130 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:55:53.100131 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 16:55:53.100135 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:55:53.100151 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 16:55:53.100155 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:55:53.100200 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0808 16:55:53.100275 (+ 75us) server_negotiation.cc:299] Negotiation successful
0808 16:55:53.100302 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 16:55:53.100546 5942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:53.096365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:37219 (local address 127.8.24.2:47195)
0808 16:55:53.096537 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:53.096565 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0808 16:55:53.096951 (+ 386us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:55:53.097129 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:55:53.097131 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:55:53.097139 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:55:53.097565 (+ 426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:53.097569 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:53.098149 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:53.098149 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:53.099006 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:53.099008 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:53.099392 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:53.099393 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:53.099542 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:53.100005 (+ 463us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 16:55:53.100005 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:55:53.100030 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 16:55:53.100051 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:55:53.100207 (+ 156us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:55:53.100212 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:55:53.100214 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 16:55:53.100272 (+ 58us) client_negotiation.cc:240] Negotiation successful
0808 16:55:53.100374 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":83,"threads_started":1}
I20260808 16:55:53.101768 5939 heartbeater.cc:347] Connected to a master server at 127.8.24.62:37219
I20260808 16:55:53.101886 5939 heartbeater.cc:464] Registering TS with master...
I20260808 16:55:53.102057 5939 heartbeater.cc:511] Master 127.8.24.62:37219 requested a full tablet report, sending...
I20260808 16:55:53.102411 5469 ts_manager.cc:194] Registered new tserver with Master: b62ec01c05734fbc98e9d87236a212bf (127.8.24.2:34469)
I20260808 16:55:53.102805 5469 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.24.2:47195
W20260808 16:55:53.178426 5943 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:55:53.178591 5943 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:55:53.178617 5943 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:55:53.179720 5943 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:55:53.179756 5943 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:55:53.179790 5943 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.3
I20260808 16:55:53.181073 5943 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:43899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786207919384338-8288-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786207919384338-8288-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786207919384338-8288-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.24.3
--webserver_port=0
--tserver_master_addrs=127.8.24.62:37219
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5943
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.3
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786207919384338-8288-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:55:53.181289 5943 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:55:53.181525 5943 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:55:53.185871 5954 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:55:53.185884 5953 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:55:53.185915 5943 server_base.cc:1034] running on GCE node
W20260808 16:55:53.185864 5956 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:55:53.186254 5943 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:55:53.186492 5943 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:55:53.187623 5943 hybrid_clock.cc:648] HybridClock initialized: now 1786208153187610 us; error 26 us; skew 500 ppm
I20260808 16:55:53.188661 5943 webserver.cc:460] Webserver started at http://127.8.24.3:41961/ using document root <none> and password file <none>
I20260808 16:55:53.188866 5943 fs_manager.cc:359] Metadata directory not provided
I20260808 16:55:53.188910 5943 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:55:53.189029 5943 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:55:53.189954 5943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786207919384338-8288-0/minicluster-data/ts-2/data/instance:
uuid: "d426c6eca8434bc09ab90081f81acb6b"
format_stamp: "Formatted at 2026-08-08 16:55:53 on dist-test-slave-6zbq"
server_key: "e6f709a3dcc55d37796b8be64afa6aea"
server_key_iv: "ebf2e4320ea97592d2f8fb68744b70c2"
server_key_version: "encryptionkey@0"
I20260808 16:55:53.190264 5943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786207919384338-8288-0/minicluster-data/ts-2/wal/instance:
uuid: "d426c6eca8434bc09ab90081f81acb6b"
format_stamp: "Formatted at 2026-08-08 16:55:53 on dist-test-slave-6zbq"
server_key: "e6f709a3dcc55d37796b8be64afa6aea"
server_key_iv: "ebf2e4320ea97592d2f8fb68744b70c2"
server_key_version: "encryptionkey@0"
I20260808 16:55:53.191390 5943 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 16:55:53.191993 5967 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:55:53.192147 5943 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 16:55:53.192231 5943 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786207919384338-8288-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786207919384338-8288-0/minicluster-data/ts-2/wal
uuid: "d426c6eca8434bc09ab90081f81acb6b"
format_stamp: "Formatted at 2026-08-08 16:55:53 on dist-test-slave-6zbq"
server_key: "e6f709a3dcc55d37796b8be64afa6aea"
server_key_iv: "ebf2e4320ea97592d2f8fb68744b70c2"
server_key_version: "encryptionkey@0"
I20260808 16:55:53.192296 5943 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786207919384338-8288-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786207919384338-8288-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786207919384338-8288-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:55:53.203317 5943 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:55:53.203528 5943 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:55:53.203636 5943 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:55:53.203812 5943 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:55:53.204088 5943 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:55:53.204120 5943 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:55:53.204145 5943 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:55:53.204162 5943 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:55:53.209244 5943 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.3:39461
I20260808 16:55:53.209278 6130 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.3:39461 every 8 connection(s)
I20260808 16:55:53.209525 5943 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786207919384338-8288-0/minicluster-data/ts-2/data/info.pb
I20260808 16:55:53.213912 5533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:53.210064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:58591 (local address 127.8.24.62:37219)
0808 16:55:53.210104 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 16:55:53.210106 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:55:53.210302 (+ 196us) server_negotiation.cc:407] Connection header received
0808 16:55:53.210758 (+ 456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:55:53.210759 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:55:53.210773 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:55:53.210833 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:55:53.211433 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:53.211910 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:53.212799 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:53.212985 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:53.213039 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:53.213670 (+ 631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:55:53.213672 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:55:53.213672 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 16:55:53.213676 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:55:53.213692 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 16:55:53.213696 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:55:53.213739 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0808 16:55:53.213811 (+ 72us) server_negotiation.cc:299] Negotiation successful
0808 16:55:53.213838 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 16:55:53.214074 6136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:53.210040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:37219 (local address 127.8.24.3:58591)
0808 16:55:53.210189 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:53.210227 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 16:55:53.210669 (+ 442us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:55:53.210901 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:55:53.210905 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:55:53.210914 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:55:53.211350 (+ 436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:53.211354 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:53.211984 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:53.211985 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:53.212713 (+ 728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:53.212716 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:53.213051 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:53.213052 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:53.213170 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:53.213549 (+ 379us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 16:55:53.213549 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:55:53.213573 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 16:55:53.213594 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:55:53.213745 (+ 151us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:55:53.213750 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:55:53.213752 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 16:55:53.213809 (+ 57us) client_negotiation.cc:240] Negotiation successful
0808 16:55:53.213900 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":64,"threads_started":1}
I20260808 16:55:53.214470 6134 heartbeater.cc:347] Connected to a master server at 127.8.24.62:37219
I20260808 16:55:53.214545 6134 heartbeater.cc:464] Registering TS with master...
I20260808 16:55:53.214730 6134 heartbeater.cc:511] Master 127.8.24.62:37219 requested a full tablet report, sending...
I20260808 16:55:53.214933 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 5943
I20260808 16:55:53.215030 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786207919384338-8288-0/minicluster-data/ts-2/wal/instance
I20260808 16:55:53.215072 5469 ts_manager.cc:194] Registered new tserver with Master: d426c6eca8434bc09ab90081f81acb6b (127.8.24.3:39461)
I20260808 16:55:53.215212 8288 external_mini_cluster.cc:1428] Setting key ccdd2389f6ef771d5341a1cc60d040c0
I20260808 16:55:53.215521 5469 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.24.3:58591
I20260808 16:55:53.215937 8288 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 16:55:53.219565 5533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:53.217188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60022 (local address 127.8.24.62:37219)
0808 16:55:53.217223 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0808 16:55:53.217224 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 16:55:53.217303 (+ 79us) server_negotiation.cc:407] Connection header received
0808 16:55:53.217350 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:55:53.217351 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:55:53.217358 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:55:53.217398 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:55:53.217509 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:53.217936 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:53.218674 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:53.218848 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:53.218892 (+ 44us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:53.219397 (+ 505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:55:53.219397 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:55:53.219398 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 16:55:53.219401 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:55:53.219409 (+ 8us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 16:55:53.219412 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:55:53.219458 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0808 16:55:53.219499 (+ 41us) server_negotiation.cc:299] Negotiation successful
0808 16:55:53.219505 (+ 6us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260808 16:55:53.220927 5469 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60022:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 16:55:53.222368 5469 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 16:55:53.228008 6153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:53.224291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.3:39461 (local address 127.0.0.1:47122)
0808 16:55:53.224855 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:53.224866 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 16:55:53.224915 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:55:53.225192 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:55:53.225193 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:55:53.225195 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:55:53.225231 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:53.225232 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:53.226262 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:53.226263 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:53.227159 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:53.227162 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:53.227746 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:53.227747 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:53.227823 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:53.227837 (+ 14us) client_negotiation.cc:769] Sending connection context
0808 16:55:53.227899 (+ 62us) client_negotiation.cc:240] Negotiation successful
0808 16:55:53.227946 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":547,"thread_start_us":58,"threads_started":1}
I20260808 16:55:53.228013 6155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:53.224398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47122 (local address 127.8.24.3:39461)
0808 16:55:53.224537 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0808 16:55:53.224541 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 16:55:53.224966 (+ 425us) server_negotiation.cc:407] Connection header received
0808 16:55:53.225001 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:55:53.225003 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:55:53.225016 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:55:53.225144 (+ 128us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:55:53.225296 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:53.226194 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:53.227218 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:53.227664 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:53.227727 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:53.227814 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 16:55:53.227910 (+ 96us) server_negotiation.cc:299] Negotiation successful
0808 16:55:53.227942 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":56,"threads_started":1}
I20260808 16:55:53.228230 6146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:53.224115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.2:34469 (local address 127.0.0.1:56968)
0808 16:55:53.224391 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:53.224408 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 16:55:53.224466 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:55:53.224761 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:55:53.224761 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:55:53.224763 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:55:53.224790 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:53.224792 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:53.225755 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:53.225756 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:53.226998 (+ 1242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:53.227001 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:53.227976 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:53.227978 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:53.228060 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:53.228064 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 16:55:53.228113 (+ 49us) client_negotiation.cc:240] Negotiation successful
0808 16:55:53.228151 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"mutex_wait_us":45,"thread_start_us":87,"threads_started":1}
I20260808 16:55:53.228440 6149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:53.224220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56968 (local address 127.8.24.2:34469)
0808 16:55:53.224616 (+ 396us) server_negotiation.cc:206] Beginning negotiation
0808 16:55:53.224620 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 16:55:53.224629 (+ 9us) server_negotiation.cc:407] Connection header received
0808 16:55:53.224665 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:55:53.224666 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:55:53.224680 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:55:53.224751 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:55:53.224858 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:53.225675 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:53.227088 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:53.227794 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:53.227861 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:53.227940 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 16:55:53.228327 (+ 387us) server_negotiation.cc:299] Negotiation successful
0808 16:55:53.228368 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":75,"threads_started":1}
I20260808 16:55:53.228580 6147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:53.224099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:40445 (local address 127.0.0.1:41484)
0808 16:55:53.224373 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:53.224408 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 16:55:53.224468 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:55:53.224652 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:55:53.224654 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:55:53.224655 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:55:53.224703 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:53.224704 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:53.225999 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:53.226000 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:53.227053 (+ 1053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:53.227055 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:53.227977 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:53.227978 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:53.228169 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:53.228172 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 16:55:53.228215 (+ 43us) client_negotiation.cc:240] Negotiation successful
0808 16:55:53.228245 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"mutex_wait_us":2,"thread_start_us":51,"threads_started":1}
I20260808 16:55:53.228657 6148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:53.224194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41484 (local address 127.8.24.1:40445)
0808 16:55:53.224369 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0808 16:55:53.224373 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 16:55:53.224474 (+ 101us) server_negotiation.cc:407] Connection header received
0808 16:55:53.224542 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:55:53.224544 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:55:53.224560 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:55:53.224653 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:55:53.224748 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:53.225920 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:53.227129 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:53.227805 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:53.227860 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:53.227940 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 16:55:53.228374 (+ 434us) server_negotiation.cc:299] Negotiation successful
0808 16:55:53.228413 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":77,"threads_started":1}
I20260808 16:55:53.228613 6037 tablet_service.cc:1467] Processing CreateTablet for tablet 39dec96f310b4784b248363adc04033a (DEFAULT_TABLE table=test-table [id=d1540beaff3b432d9aea4b691f51f592]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:55:53.228955 6037 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39dec96f310b4784b248363adc04033a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:55:53.229085 5850 tablet_service.cc:1467] Processing CreateTablet for tablet 39dec96f310b4784b248363adc04033a (DEFAULT_TABLE table=test-table [id=d1540beaff3b432d9aea4b691f51f592]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:55:53.229162 5647 tablet_service.cc:1467] Processing CreateTablet for tablet 39dec96f310b4784b248363adc04033a (DEFAULT_TABLE table=test-table [id=d1540beaff3b432d9aea4b691f51f592]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:55:53.229395 5647 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39dec96f310b4784b248363adc04033a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:55:53.229398 5850 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39dec96f310b4784b248363adc04033a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:55:53.231724 6158 tablet_bootstrap.cc:492] T 39dec96f310b4784b248363adc04033a P 0140c1c94154476eb82c96d55cc05e6b: Bootstrap starting.
I20260808 16:55:53.232007 6159 tablet_bootstrap.cc:492] T 39dec96f310b4784b248363adc04033a P d426c6eca8434bc09ab90081f81acb6b: Bootstrap starting.
I20260808 16:55:53.232357 6158 tablet_bootstrap.cc:654] T 39dec96f310b4784b248363adc04033a P 0140c1c94154476eb82c96d55cc05e6b: Neither blocks nor log segments found. Creating new log.
I20260808 16:55:53.232405 6161 tablet_bootstrap.cc:492] T 39dec96f310b4784b248363adc04033a P b62ec01c05734fbc98e9d87236a212bf: Bootstrap starting.
I20260808 16:55:53.232548 6159 tablet_bootstrap.cc:654] T 39dec96f310b4784b248363adc04033a P d426c6eca8434bc09ab90081f81acb6b: Neither blocks nor log segments found. Creating new log.
I20260808 16:55:53.232650 6158 log.cc:824] T 39dec96f310b4784b248363adc04033a P 0140c1c94154476eb82c96d55cc05e6b: Log is configured to *not* fsync() on all Append() calls
I20260808 16:55:53.232828 6159 log.cc:824] T 39dec96f310b4784b248363adc04033a P d426c6eca8434bc09ab90081f81acb6b: Log is configured to *not* fsync() on all Append() calls
I20260808 16:55:53.233006 6161 tablet_bootstrap.cc:654] T 39dec96f310b4784b248363adc04033a P b62ec01c05734fbc98e9d87236a212bf: Neither blocks nor log segments found. Creating new log.
I20260808 16:55:53.233261 6161 log.cc:824] T 39dec96f310b4784b248363adc04033a P b62ec01c05734fbc98e9d87236a212bf: Log is configured to *not* fsync() on all Append() calls
I20260808 16:55:53.233402 6158 tablet_bootstrap.cc:492] T 39dec96f310b4784b248363adc04033a P 0140c1c94154476eb82c96d55cc05e6b: No bootstrap required, opened a new log
I20260808 16:55:53.233458 6158 ts_tablet_manager.cc:1397] T 39dec96f310b4784b248363adc04033a P 0140c1c94154476eb82c96d55cc05e6b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 16:55:53.233839 6161 tablet_bootstrap.cc:492] T 39dec96f310b4784b248363adc04033a P b62ec01c05734fbc98e9d87236a212bf: No bootstrap required, opened a new log
I20260808 16:55:53.233891 6161 ts_tablet_manager.cc:1397] T 39dec96f310b4784b248363adc04033a P b62ec01c05734fbc98e9d87236a212bf: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 16:55:53.234908 6159 tablet_bootstrap.cc:492] T 39dec96f310b4784b248363adc04033a P d426c6eca8434bc09ab90081f81acb6b: No bootstrap required, opened a new log
I20260808 16:55:53.234975 6159 ts_tablet_manager.cc:1397] T 39dec96f310b4784b248363adc04033a P d426c6eca8434bc09ab90081f81acb6b: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260808 16:55:53.235370 6158 raft_consensus.cc:348] T 39dec96f310b4784b248363adc04033a P 0140c1c94154476eb82c96d55cc05e6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0140c1c94154476eb82c96d55cc05e6b" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 40445 } } peers { permanent_uuid: "b62ec01c05734fbc98e9d87236a212bf" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 34469 } } peers { permanent_uuid: "d426c6eca8434bc09ab90081f81acb6b" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 39461 } }
I20260808 16:55:53.235489 6158 raft_consensus.cc:374] T 39dec96f310b4784b248363adc04033a P 0140c1c94154476eb82c96d55cc05e6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:55:53.235522 6158 raft_consensus.cc:729] T 39dec96f310b4784b248363adc04033a P 0140c1c94154476eb82c96d55cc05e6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0140c1c94154476eb82c96d55cc05e6b, State: Initialized, Role: FOLLOWER
I20260808 16:55:53.235623 6158 consensus_queue.cc:260] T 39dec96f310b4784b248363adc04033a P 0140c1c94154476eb82c96d55cc05e6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0140c1c94154476eb82c96d55cc05e6b" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 40445 } } peers { permanent_uuid: "b62ec01c05734fbc98e9d87236a212bf" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 34469 } } peers { permanent_uuid: "d426c6eca8434bc09ab90081f81acb6b" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 39461 } }
I20260808 16:55:53.235854 6158 ts_tablet_manager.cc:1428] T 39dec96f310b4784b248363adc04033a P 0140c1c94154476eb82c96d55cc05e6b: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 16:55:53.235972 5745 heartbeater.cc:503] Master 127.8.24.62:37219 was elected leader, sending a full tablet report...
I20260808 16:55:53.236236 6161 raft_consensus.cc:348] T 39dec96f310b4784b248363adc04033a P b62ec01c05734fbc98e9d87236a212bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0140c1c94154476eb82c96d55cc05e6b" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 40445 } } peers { permanent_uuid: "b62ec01c05734fbc98e9d87236a212bf" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 34469 } } peers { permanent_uuid: "d426c6eca8434bc09ab90081f81acb6b" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 39461 } }
I20260808 16:55:53.236389 6161 raft_consensus.cc:374] T 39dec96f310b4784b248363adc04033a P b62ec01c05734fbc98e9d87236a212bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:55:53.236431 6161 raft_consensus.cc:729] T 39dec96f310b4784b248363adc04033a P b62ec01c05734fbc98e9d87236a212bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b62ec01c05734fbc98e9d87236a212bf, State: Initialized, Role: FOLLOWER
I20260808 16:55:53.236539 6161 consensus_queue.cc:260] T 39dec96f310b4784b248363adc04033a P b62ec01c05734fbc98e9d87236a212bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0140c1c94154476eb82c96d55cc05e6b" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 40445 } } peers { permanent_uuid: "b62ec01c05734fbc98e9d87236a212bf" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 34469 } } peers { permanent_uuid: "d426c6eca8434bc09ab90081f81acb6b" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 39461 } }
I20260808 16:55:53.236792 6161 ts_tablet_manager.cc:1428] T 39dec96f310b4784b248363adc04033a P b62ec01c05734fbc98e9d87236a212bf: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 16:55:53.236912 5939 heartbeater.cc:503] Master 127.8.24.62:37219 was elected leader, sending a full tablet report...
I20260808 16:55:53.237646 6159 raft_consensus.cc:348] T 39dec96f310b4784b248363adc04033a P d426c6eca8434bc09ab90081f81acb6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0140c1c94154476eb82c96d55cc05e6b" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 40445 } } peers { permanent_uuid: "b62ec01c05734fbc98e9d87236a212bf" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 34469 } } peers { permanent_uuid: "d426c6eca8434bc09ab90081f81acb6b" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 39461 } }
I20260808 16:55:53.237788 6159 raft_consensus.cc:374] T 39dec96f310b4784b248363adc04033a P d426c6eca8434bc09ab90081f81acb6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:55:53.237824 6159 raft_consensus.cc:729] T 39dec96f310b4784b248363adc04033a P d426c6eca8434bc09ab90081f81acb6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d426c6eca8434bc09ab90081f81acb6b, State: Initialized, Role: FOLLOWER
I20260808 16:55:53.237926 6159 consensus_queue.cc:260] T 39dec96f310b4784b248363adc04033a P d426c6eca8434bc09ab90081f81acb6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0140c1c94154476eb82c96d55cc05e6b" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 40445 } } peers { permanent_uuid: "b62ec01c05734fbc98e9d87236a212bf" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 34469 } } peers { permanent_uuid: "d426c6eca8434bc09ab90081f81acb6b" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 39461 } }
I20260808 16:55:53.238152 6159 ts_tablet_manager.cc:1428] T 39dec96f310b4784b248363adc04033a P d426c6eca8434bc09ab90081f81acb6b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 16:55:53.238169 6134 heartbeater.cc:503] Master 127.8.24.62:37219 was elected leader, sending a full tablet report...
I20260808 16:55:53.331789 6166 raft_consensus.cc:482] T 39dec96f310b4784b248363adc04033a P 0140c1c94154476eb82c96d55cc05e6b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 16:55:53.331884 6166 raft_consensus.cc:504] T 39dec96f310b4784b248363adc04033a P 0140c1c94154476eb82c96d55cc05e6b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0140c1c94154476eb82c96d55cc05e6b" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 40445 } } peers { permanent_uuid: "b62ec01c05734fbc98e9d87236a212bf" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 34469 } } peers { permanent_uuid: "d426c6eca8434bc09ab90081f81acb6b" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 39461 } }
I20260808 16:55:53.332175 6166 leader_election.cc:288] T 39dec96f310b4784b248363adc04033a P 0140c1c94154476eb82c96d55cc05e6b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b62ec01c05734fbc98e9d87236a212bf (127.8.24.2:34469), d426c6eca8434bc09ab90081f81acb6b (127.8.24.3:39461)
I20260808 16:55:53.334784 5748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:53.332307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.2:34469 (local address 127.8.24.1:43031)
0808 16:55:53.332355 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:53.332366 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 16:55:53.332403 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:55:53.332532 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:55:53.332533 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:55:53.332535 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:55:53.332587 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:53.332588 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:53.333266 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:53.333267 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:53.334021 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:53.334023 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:53.334574 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:53.334575 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:53.334638 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:53.334642 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 16:55:53.334692 (+ 50us) client_negotiation.cc:240] Negotiation successful
0808 16:55:53.334732 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260808 16:55:53.334791 6149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:53.332377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:43031 (local address 127.8.24.2:34469)
0808 16:55:53.332419 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 16:55:53.332421 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:55:53.332429 (+ 8us) server_negotiation.cc:407] Connection header received
0808 16:55:53.332454 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:55:53.332455 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:55:53.332463 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:55:53.332521 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:55:53.332655 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:53.333195 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:53.334103 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:53.334508 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:53.334557 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:53.334565 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 16:55:53.334697 (+ 132us) server_negotiation.cc:299] Negotiation successful
0808 16:55:53.334730 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 16:55:53.335075 6169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:53.332400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.3:39461 (local address 127.8.24.1:57323)
0808 16:55:53.332552 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:53.332562 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 16:55:53.332610 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:55:53.332738 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:55:53.332739 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:55:53.332741 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:55:53.332805 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:53.332807 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:53.333384 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:53.333385 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:53.334138 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:53.334140 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:53.334685 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:53.334686 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:53.334771 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:53.334775 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 16:55:53.334965 (+ 190us) client_negotiation.cc:240] Negotiation successful
0808 16:55:53.335002 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":82,"threads_started":1}
I20260808 16:55:53.335086 6155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:53.332463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:57323 (local address 127.8.24.3:39461)
0808 16:55:53.332510 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 16:55:53.332511 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 16:55:53.332615 (+ 104us) server_negotiation.cc:407] Connection header received
0808 16:55:53.332674 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:55:53.332675 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:55:53.332691 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:55:53.332721 (+ 30us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:55:53.332857 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:53.333311 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:53.334226 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:53.334607 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:53.334670 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:53.334679 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 16:55:53.334990 (+ 311us) server_negotiation.cc:299] Negotiation successful
0808 16:55:53.335032 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 16:55:53.335083 5873 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39dec96f310b4784b248363adc04033a" candidate_uuid: "0140c1c94154476eb82c96d55cc05e6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b62ec01c05734fbc98e9d87236a212bf" is_pre_election: true
I20260808 16:55:53.335210 5873 raft_consensus.cc:2454] T 39dec96f310b4784b248363adc04033a P b62ec01c05734fbc98e9d87236a212bf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0140c1c94154476eb82c96d55cc05e6b in term 0.
I20260808 16:55:53.335415 6067 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39dec96f310b4784b248363adc04033a" candidate_uuid: "0140c1c94154476eb82c96d55cc05e6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d426c6eca8434bc09ab90081f81acb6b" is_pre_election: true
I20260808 16:55:53.335443 5583 leader_election.cc:302] T 39dec96f310b4784b248363adc04033a P 0140c1c94154476eb82c96d55cc05e6b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0140c1c94154476eb82c96d55cc05e6b, b62ec01c05734fbc98e9d87236a212bf; no voters:
I20260808 16:55:53.335529 6067 raft_consensus.cc:2454] T 39dec96f310b4784b248363adc04033a P d426c6eca8434bc09ab90081f81acb6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0140c1c94154476eb82c96d55cc05e6b in term 0.
I20260808 16:55:53.335546 6166 raft_consensus.cc:2781] T 39dec96f310b4784b248363adc04033a P 0140c1c94154476eb82c96d55cc05e6b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 16:55:53.335592 6166 raft_consensus.cc:482] T 39dec96f310b4784b248363adc04033a P 0140c1c94154476eb82c96d55cc05e6b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 16:55:53.335618 6166 raft_consensus.cc:3037] T 39dec96f310b4784b248363adc04033a P 0140c1c94154476eb82c96d55cc05e6b [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:55:53.336226 6166 raft_consensus.cc:504] T 39dec96f310b4784b248363adc04033a P 0140c1c94154476eb82c96d55cc05e6b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0140c1c94154476eb82c96d55cc05e6b" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 40445 } } peers { permanent_uuid: "b62ec01c05734fbc98e9d87236a212bf" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 34469 } } peers { permanent_uuid: "d426c6eca8434bc09ab90081f81acb6b" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 39461 } }
I20260808 16:55:53.336329 6166 leader_election.cc:288] T 39dec96f310b4784b248363adc04033a P 0140c1c94154476eb82c96d55cc05e6b [CANDIDATE]: Term 1 election: Requested vote from peers b62ec01c05734fbc98e9d87236a212bf (127.8.24.2:34469), d426c6eca8434bc09ab90081f81acb6b (127.8.24.3:39461)
I20260808 16:55:53.336478 5873 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39dec96f310b4784b248363adc04033a" candidate_uuid: "0140c1c94154476eb82c96d55cc05e6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b62ec01c05734fbc98e9d87236a212bf"
I20260808 16:55:53.336489 6067 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39dec96f310b4784b248363adc04033a" candidate_uuid: "0140c1c94154476eb82c96d55cc05e6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d426c6eca8434bc09ab90081f81acb6b"
I20260808 16:55:53.336548 6067 raft_consensus.cc:3037] T 39dec96f310b4784b248363adc04033a P d426c6eca8434bc09ab90081f81acb6b [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:55:53.336544 5873 raft_consensus.cc:3037] T 39dec96f310b4784b248363adc04033a P b62ec01c05734fbc98e9d87236a212bf [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:55:53.337160 6067 raft_consensus.cc:2454] T 39dec96f310b4784b248363adc04033a P d426c6eca8434bc09ab90081f81acb6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0140c1c94154476eb82c96d55cc05e6b in term 1.
I20260808 16:55:53.337198 5873 raft_consensus.cc:2454] T 39dec96f310b4784b248363adc04033a P b62ec01c05734fbc98e9d87236a212bf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0140c1c94154476eb82c96d55cc05e6b in term 1.
I20260808 16:55:53.337328 5583 leader_election.cc:302] T 39dec96f310b4784b248363adc04033a P 0140c1c94154476eb82c96d55cc05e6b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0140c1c94154476eb82c96d55cc05e6b, b62ec01c05734fbc98e9d87236a212bf; no voters:
I20260808 16:55:53.337414 6166 raft_consensus.cc:2781] T 39dec96f310b4784b248363adc04033a P 0140c1c94154476eb82c96d55cc05e6b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:55:53.337535 6166 raft_consensus.cc:686] T 39dec96f310b4784b248363adc04033a P 0140c1c94154476eb82c96d55cc05e6b [term 1 LEADER]: Becoming Leader. State: Replica: 0140c1c94154476eb82c96d55cc05e6b, State: Running, Role: LEADER
I20260808 16:55:53.337607 6166 consensus_queue.cc:237] T 39dec96f310b4784b248363adc04033a P 0140c1c94154476eb82c96d55cc05e6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0140c1c94154476eb82c96d55cc05e6b" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 40445 } } peers { permanent_uuid: "b62ec01c05734fbc98e9d87236a212bf" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 34469 } } peers { permanent_uuid: "d426c6eca8434bc09ab90081f81acb6b" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 39461 } }
I20260808 16:55:53.338295 5467 catalog_manager.cc:5495] T 39dec96f310b4784b248363adc04033a P 0140c1c94154476eb82c96d55cc05e6b reported cstate change: term changed from 0 to 1, leader changed from <none> to 0140c1c94154476eb82c96d55cc05e6b (127.8.24.1). New cstate: current_term: 1 leader_uuid: "0140c1c94154476eb82c96d55cc05e6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0140c1c94154476eb82c96d55cc05e6b" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 40445 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b62ec01c05734fbc98e9d87236a212bf" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 34469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d426c6eca8434bc09ab90081f81acb6b" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 39461 } health_report { overall_health: UNKNOWN } } }
I20260808 16:55:53.343232 6148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:53.341248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41496 (local address 127.8.24.1:40445)
0808 16:55:53.341305 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 16:55:53.341307 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:55:53.341332 (+ 25us) server_negotiation.cc:407] Connection header received
0808 16:55:53.341359 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:55:53.341359 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:55:53.341374 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:55:53.341426 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 16:55:53.341527 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:53.341995 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:53.342671 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:53.342839 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:53.342885 (+ 46us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:53.342993 (+ 108us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 16:55:53.343066 (+ 73us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 16:55:53.343132 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 16:55:53.343152 (+ 20us) server_negotiation.cc:299] Negotiation successful
0808 16:55:53.343158 (+ 6us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 16:55:53.344655 5873 raft_consensus.cc:1261] T 39dec96f310b4784b248363adc04033a P b62ec01c05734fbc98e9d87236a212bf [term 1 FOLLOWER]: Refusing update from remote peer 0140c1c94154476eb82c96d55cc05e6b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:55:53.344655 6067 raft_consensus.cc:1261] T 39dec96f310b4784b248363adc04033a P d426c6eca8434bc09ab90081f81acb6b [term 1 FOLLOWER]: Refusing update from remote peer 0140c1c94154476eb82c96d55cc05e6b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:55:53.344969 6171 consensus_queue.cc:1035] T 39dec96f310b4784b248363adc04033a P 0140c1c94154476eb82c96d55cc05e6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d426c6eca8434bc09ab90081f81acb6b" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 39461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:55:53.345085 6166 consensus_queue.cc:1035] T 39dec96f310b4784b248363adc04033a P 0140c1c94154476eb82c96d55cc05e6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b62ec01c05734fbc98e9d87236a212bf" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 34469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 16:55:53.346196 5941 tablet.cc:2369] T 39dec96f310b4784b248363adc04033a P b62ec01c05734fbc98e9d87236a212bf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 16:55:53.347427 6176 mvcc.cc:205] Tried to move back new op lower bound from 7316308596097699840 to 7316308596071350272. Current Snapshot: MvccSnapshot[applied={T|T < 7316308596097699840}]
I20260808 16:55:53.347527 6177 mvcc.cc:205] Tried to move back new op lower bound from 7316308596097699840 to 7316308596071350272. Current Snapshot: MvccSnapshot[applied={T|T < 7316308596097699840}]
I20260808 16:55:53.349516 5467 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60022:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 16:55:53.349601 5467 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60022:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 16:55:53.350207 5467 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dd603eb005ae4aa6bd01c11281c87a0f: Sending DeleteTablet for 3 replicas of tablet 39dec96f310b4784b248363adc04033a
I20260808 16:55:53.350533 5647 tablet_service.cc:1514] Processing DeleteTablet for tablet 39dec96f310b4784b248363adc04033a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:55:53 UTC) from {username='slave'} at 127.0.0.1:41484
I20260808 16:55:53.350598 5850 tablet_service.cc:1514] Processing DeleteTablet for tablet 39dec96f310b4784b248363adc04033a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:55:53 UTC) from {username='slave'} at 127.0.0.1:56968
I20260808 16:55:53.350637 6037 tablet_service.cc:1514] Processing DeleteTablet for tablet 39dec96f310b4784b248363adc04033a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:55:53 UTC) from {username='slave'} at 127.0.0.1:47122
I20260808 16:55:53.350797 6191 tablet_replica.cc:330] T 39dec96f310b4784b248363adc04033a P b62ec01c05734fbc98e9d87236a212bf: stopping tablet replica
I20260808 16:55:53.350896 6191 raft_consensus.cc:2229] T 39dec96f310b4784b248363adc04033a P b62ec01c05734fbc98e9d87236a212bf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 16:55:53.350996 6191 raft_consensus.cc:2258] T 39dec96f310b4784b248363adc04033a P b62ec01c05734fbc98e9d87236a212bf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:55:53.351052 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 5563
I20260808 16:55:53.353647 6191 ts_tablet_manager.cc:1905] T 39dec96f310b4784b248363adc04033a P b62ec01c05734fbc98e9d87236a212bf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:55:53.357327 6191 ts_tablet_manager.cc:1918] T 39dec96f310b4784b248363adc04033a P b62ec01c05734fbc98e9d87236a212bf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 16:55:53.357378 6191 log.cc:1196] T 39dec96f310b4784b248363adc04033a P b62ec01c05734fbc98e9d87236a212bf: Deleting WAL directory at /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786207919384338-8288-0/minicluster-data/ts-1/wal/wals/39dec96f310b4784b248363adc04033a
I20260808 16:55:53.357543 6191 ts_tablet_manager.cc:1939] T 39dec96f310b4784b248363adc04033a P b62ec01c05734fbc98e9d87236a212bf: Deleting consensus metadata
I20260808 16:55:53.358616 5446 catalog_manager.cc:4841] TS b62ec01c05734fbc98e9d87236a212bf (127.8.24.2:34469): tablet 39dec96f310b4784b248363adc04033a (table test-table [id=d1540beaff3b432d9aea4b691f51f592]) successfully deleted
I20260808 16:55:53.360018 6192 tablet_replica.cc:330] T 39dec96f310b4784b248363adc04033a P d426c6eca8434bc09ab90081f81acb6b: stopping tablet replica
I20260808 16:55:53.360105 6192 raft_consensus.cc:2229] T 39dec96f310b4784b248363adc04033a P d426c6eca8434bc09ab90081f81acb6b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 16:55:53.360198 6192 raft_consensus.cc:2258] T 39dec96f310b4784b248363adc04033a P d426c6eca8434bc09ab90081f81acb6b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:55:53.360555 6192 ts_tablet_manager.cc:1905] T 39dec96f310b4784b248363adc04033a P d426c6eca8434bc09ab90081f81acb6b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:55:53.361625 6192 ts_tablet_manager.cc:1918] T 39dec96f310b4784b248363adc04033a P d426c6eca8434bc09ab90081f81acb6b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 16:55:53.361668 6192 log.cc:1196] T 39dec96f310b4784b248363adc04033a P d426c6eca8434bc09ab90081f81acb6b: Deleting WAL directory at /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786207919384338-8288-0/minicluster-data/ts-2/wal/wals/39dec96f310b4784b248363adc04033a
I20260808 16:55:53.361837 6192 ts_tablet_manager.cc:1939] T 39dec96f310b4784b248363adc04033a P d426c6eca8434bc09ab90081f81acb6b: Deleting consensus metadata
I20260808 16:55:53.362141 5446 catalog_manager.cc:4841] TS d426c6eca8434bc09ab90081f81acb6b (127.8.24.3:39461): tablet 39dec96f310b4784b248363adc04033a (table test-table [id=d1540beaff3b432d9aea4b691f51f592]) successfully deleted
W20260808 16:55:53.364277 5447 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.24.1:40445 (error 108)
I20260808 16:55:53.364590 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 5749
I20260808 16:55:53.364717 6147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:53.364593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:40445 (local address 127.0.0.1:41498)
0808 16:55:53.364641 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:53.364692 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.1:40445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260808 16:55:53.369904 5447 catalog_manager.cc:4584] TS 0140c1c94154476eb82c96d55cc05e6b (127.8.24.1:40445): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 39dec96f310b4784b248363adc04033a: Network error: Client connection negotiation failed: client connection to 127.8.24.1:40445: connect: Connection refused (error 111)
I20260808 16:55:53.375061 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 5943
I20260808 16:55:53.391047 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 5418
2026-08-08T16:55:53Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (713 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786207919384338-8288-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 16:55:53 dist-test-slave-6zbq krb5kdc[6197](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 16:55:53 dist-test-slave-6zbq krb5kdc[6197](info): set up 2 sockets
Aug 08 16:55:53 dist-test-slave-6zbq krb5kdc[6197](info): commencing operation
krb5kdc: starting...
W20260808 16:55:54.524078 8288 mini_kdc.cc:121] Time spent starting KDC: real 1.108s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786207919384338-8288-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786207919384338-8288-0/krb5kdc/test-user.keytab.
Aug 08 16:55:54 dist-test-slave-6zbq krb5kdc[6197](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208154, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T16:55:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T16:55:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.24.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
I20260808 16:55:54.632421 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786207919384338-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786207919384338-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786207919384338-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:42089
--webserver_interface=127.8.24.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:44977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:42089
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:55:54.711239 6220 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:55:54.711437 6220 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:55:54.711467 6220 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:55:54.712569 6220 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 16:55:54.712610 6220 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:55:54.712627 6220 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:55:54.712643 6220 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 16:55:54.712658 6220 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 16:55:54.714062 6220 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:44977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786207919384338-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786207919384338-8288-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:42089
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:42089
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.24.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6220
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786207919384338-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:55:54.714308 6220 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:55:54.714515 6220 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:55:54.718981 6232 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:55:54.719012 6236 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:55:54.719018 6230 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:55:54.719164 6220 server_base.cc:1034] running on GCE node
I20260808 16:55:54.719512 6220 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:55:54.719784 6220 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:55:54.720932 6220 hybrid_clock.cc:648] HybridClock initialized: now 1786208154720919 us; error 27 us; skew 500 ppm
Aug 08 16:55:54 dist-test-slave-6zbq krb5kdc[6197](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208154, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:55:54.726658 6220 init.cc:377] Logged in from keytab as kudu/127.8.24.62@KRBTEST.COM (short username kudu)
I20260808 16:55:54.727029 6220 webserver.cc:460] Webserver started at http://127.8.24.62:35269/ using document root <none> and password file <none>
I20260808 16:55:54.727216 6220 fs_manager.cc:359] Metadata directory not provided
I20260808 16:55:54.727257 6220 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:55:54.727371 6220 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:55:54.728330 6220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786207919384338-8288-0/minicluster-data/master-0/data/instance:
uuid: "c43eb7ef565944e08406ef06c634326d"
format_stamp: "Formatted at 2026-08-08 16:55:54 on dist-test-slave-6zbq"
server_key: "95334d54f0eea251968557ce1a53b7f3"
server_key_iv: "70411a42dce16de6c0a22fc6a3daa327"
server_key_version: "encryptionkey@0"
I20260808 16:55:54.728647 6220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786207919384338-8288-0/minicluster-data/master-0/wal/instance:
uuid: "c43eb7ef565944e08406ef06c634326d"
format_stamp: "Formatted at 2026-08-08 16:55:54 on dist-test-slave-6zbq"
server_key: "95334d54f0eea251968557ce1a53b7f3"
server_key_iv: "70411a42dce16de6c0a22fc6a3daa327"
server_key_version: "encryptionkey@0"
I20260808 16:55:54.729781 6220 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 16:55:54.730392 6247 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:55:54.730525 6220 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 16:55:54.730608 6220 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786207919384338-8288-0/minicluster-data/master-0/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786207919384338-8288-0/minicluster-data/master-0/wal
uuid: "c43eb7ef565944e08406ef06c634326d"
format_stamp: "Formatted at 2026-08-08 16:55:54 on dist-test-slave-6zbq"
server_key: "95334d54f0eea251968557ce1a53b7f3"
server_key_iv: "70411a42dce16de6c0a22fc6a3daa327"
server_key_version: "encryptionkey@0"
I20260808 16:55:54.730676 6220 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786207919384338-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786207919384338-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786207919384338-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:55:54.738056 6220 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:55:54.739615 6220 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:55:54.739745 6220 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:55:54.743350 6220 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.62:42089
I20260808 16:55:54.743384 6314 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.62:42089 every 8 connection(s)
I20260808 16:55:54.743665 6220 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
I20260808 16:55:54.744446 6315 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:55:54.746213 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 6220
I20260808 16:55:54.746327 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786207919384338-8288-0/minicluster-data/master-0/wal/instance
I20260808 16:55:54.746541 8288 external_mini_cluster.cc:1428] Setting key bf19677edac4887bbcaf7de430799dd9
I20260808 16:55:54.746584 6315 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c43eb7ef565944e08406ef06c634326d: Bootstrap starting.
I20260808 16:55:54.747282 6315 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c43eb7ef565944e08406ef06c634326d: Neither blocks nor log segments found. Creating new log.
I20260808 16:55:54.747588 6315 log.cc:824] T 00000000000000000000000000000000 P c43eb7ef565944e08406ef06c634326d: Log is configured to *not* fsync() on all Append() calls
I20260808 16:55:54.748255 6315 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c43eb7ef565944e08406ef06c634326d: No bootstrap required, opened a new log
I20260808 16:55:54.750480 6315 raft_consensus.cc:348] T 00000000000000000000000000000000 P c43eb7ef565944e08406ef06c634326d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c43eb7ef565944e08406ef06c634326d" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 42089 } }
I20260808 16:55:54.750643 6315 raft_consensus.cc:374] T 00000000000000000000000000000000 P c43eb7ef565944e08406ef06c634326d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:55:54.750687 6315 raft_consensus.cc:729] T 00000000000000000000000000000000 P c43eb7ef565944e08406ef06c634326d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43eb7ef565944e08406ef06c634326d, State: Initialized, Role: FOLLOWER
I20260808 16:55:54.750810 6315 consensus_queue.cc:260] T 00000000000000000000000000000000 P c43eb7ef565944e08406ef06c634326d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c43eb7ef565944e08406ef06c634326d" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 42089 } }
I20260808 16:55:54.750883 6315 raft_consensus.cc:388] T 00000000000000000000000000000000 P c43eb7ef565944e08406ef06c634326d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 16:55:54.750918 6315 raft_consensus.cc:482] T 00000000000000000000000000000000 P c43eb7ef565944e08406ef06c634326d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 16:55:54.750960 6315 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c43eb7ef565944e08406ef06c634326d [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:55:54.751582 6315 raft_consensus.cc:504] T 00000000000000000000000000000000 P c43eb7ef565944e08406ef06c634326d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c43eb7ef565944e08406ef06c634326d" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 42089 } }
I20260808 16:55:54.751693 6315 leader_election.cc:302] T 00000000000000000000000000000000 P c43eb7ef565944e08406ef06c634326d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c43eb7ef565944e08406ef06c634326d; no voters:
I20260808 16:55:54.751854 6315 leader_election.cc:288] T 00000000000000000000000000000000 P c43eb7ef565944e08406ef06c634326d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 16:55:54.751883 6323 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c43eb7ef565944e08406ef06c634326d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:55:54.752007 6323 raft_consensus.cc:686] T 00000000000000000000000000000000 P c43eb7ef565944e08406ef06c634326d [term 1 LEADER]: Becoming Leader. State: Replica: c43eb7ef565944e08406ef06c634326d, State: Running, Role: LEADER
I20260808 16:55:54.752100 6315 sys_catalog.cc:564] T 00000000000000000000000000000000 P c43eb7ef565944e08406ef06c634326d [sys.catalog]: configured and running, proceeding with master startup.
I20260808 16:55:54.752089 6323 consensus_queue.cc:237] T 00000000000000000000000000000000 P c43eb7ef565944e08406ef06c634326d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c43eb7ef565944e08406ef06c634326d" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 42089 } }
I20260808 16:55:54.752388 6323 sys_catalog.cc:455] T 00000000000000000000000000000000 P c43eb7ef565944e08406ef06c634326d [sys.catalog]: SysCatalogTable state changed. Reason: New leader c43eb7ef565944e08406ef06c634326d. Latest consensus state: current_term: 1 leader_uuid: "c43eb7ef565944e08406ef06c634326d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c43eb7ef565944e08406ef06c634326d" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 42089 } } }
I20260808 16:55:54.752457 6323 sys_catalog.cc:458] T 00000000000000000000000000000000 P c43eb7ef565944e08406ef06c634326d [sys.catalog]: This master's current role is: LEADER
I20260808 16:55:54.752619 6325 sys_catalog.cc:455] T 00000000000000000000000000000000 P c43eb7ef565944e08406ef06c634326d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c43eb7ef565944e08406ef06c634326d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c43eb7ef565944e08406ef06c634326d" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 42089 } } }
I20260808 16:55:54.752727 6325 sys_catalog.cc:458] T 00000000000000000000000000000000 P c43eb7ef565944e08406ef06c634326d [sys.catalog]: This master's current role is: LEADER
Aug 08 16:55:54 dist-test-slave-6zbq krb5kdc[6197](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208154, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
W20260808 16:55:54.753831 6349 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c43eb7ef565944e08406ef06c634326d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 16:55:54.753881 6349 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 16:55:54.754089 6328 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 16:55:54.754267 6328 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 16:55:54.756311 6328 catalog_manager.cc:1347] Generated new cluster ID: 54229b25f64b44549a50b0d0bd9ef089
I20260808 16:55:54.756357 6328 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 16:55:54.757061 6320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:54.746953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35536 (local address 127.8.24.62:42089)
0808 16:55:54.747109 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 16:55:54.747118 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 16:55:54.747154 (+ 36us) server_negotiation.cc:407] Connection header received
0808 16:55:54.747704 (+ 550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:55:54.747713 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:55:54.747786 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:55:54.747906 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:55:54.748510 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:54.749816 (+ 1306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:54.750549 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:54.751119 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:54.751227 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:54.753895 (+ 2668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:55:54.753899 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:55:54.753899 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:55:54.753910 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:55:54.755412 (+ 1502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:55:54.755825 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:55:54.755828 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:55:54.755828 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:55:54.755923 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:55:54.756134 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:55:54.756135 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:55:54.756136 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:55:54.756437 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:55:54.756592 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0808 16:55:54.756713 (+ 121us) server_negotiation.cc:299] Negotiation successful
0808 16:55:54.756851 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":74,"threads_started":1}
I20260808 16:55:54.759837 6328 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 16:55:54.760509 6328 catalog_manager.cc:1504] Loading token signing keys...
I20260808 16:55:54.769577 6328 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c43eb7ef565944e08406ef06c634326d: Generated new TSK 0
I20260808 16:55:54.769735 6328 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.24.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.1.keytab.
Entry for principal kudu/127.8.24.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.1.keytab.
I20260808 16:55:54.795042 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786207919384338-8288-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786207919384338-8288-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786207919384338-8288-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786207919384338-8288-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.1:0
--local_ip_for_outbound_sockets=127.8.24.1
--webserver_interface=127.8.24.1
--webserver_port=0
--tserver_master_addrs=127.8.24.62:42089
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.1.keytab
--principal=kudu/127.8.24.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:44977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:55:54.872781 6360 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:55:54.872952 6360 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:55:54.872974 6360 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:55:54.874055 6360 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:55:54.874094 6360 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:55:54.874121 6360 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.1
I20260808 16:55:54.875557 6360 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:44977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786207919384338-8288-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786207919384338-8288-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.1.keytab
--principal=kudu/127.8.24.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786207919384338-8288-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.24.1
--webserver_port=0
--tserver_master_addrs=127.8.24.62:42089
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6360
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.1
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786207919384338-8288-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:55:54.875769 6360 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:55:54.875967 6360 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:55:54.880515 6379 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:55:54.880525 6376 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:55:54.880708 6375 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:55:54.880719 6360 server_base.cc:1034] running on GCE node
I20260808 16:55:54.880925 6360 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:55:54.881129 6360 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:55:54.882272 6360 hybrid_clock.cc:648] HybridClock initialized: now 1786208154882257 us; error 33 us; skew 500 ppm
Aug 08 16:55:54 dist-test-slave-6zbq krb5kdc[6197](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208154, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:55:54.884608 6360 init.cc:377] Logged in from keytab as kudu/127.8.24.1@KRBTEST.COM (short username kudu)
I20260808 16:55:54.885016 6360 webserver.cc:460] Webserver started at http://127.8.24.1:36483/ using document root <none> and password file <none>
I20260808 16:55:54.885221 6360 fs_manager.cc:359] Metadata directory not provided
I20260808 16:55:54.885267 6360 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:55:54.885387 6360 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:55:54.886322 6360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786207919384338-8288-0/minicluster-data/ts-0/data/instance:
uuid: "440bde2bb1ba4f829a04dc0cac541751"
format_stamp: "Formatted at 2026-08-08 16:55:54 on dist-test-slave-6zbq"
server_key: "01b377a17de586a89334ebb71d545d05"
server_key_iv: "386dd0daadefa848104de7bc1bd45887"
server_key_version: "encryptionkey@0"
I20260808 16:55:54.886686 6360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786207919384338-8288-0/minicluster-data/ts-0/wal/instance:
uuid: "440bde2bb1ba4f829a04dc0cac541751"
format_stamp: "Formatted at 2026-08-08 16:55:54 on dist-test-slave-6zbq"
server_key: "01b377a17de586a89334ebb71d545d05"
server_key_iv: "386dd0daadefa848104de7bc1bd45887"
server_key_version: "encryptionkey@0"
I20260808 16:55:54.887831 6360 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 16:55:54.888427 6391 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:55:54.888572 6360 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 16:55:54.888674 6360 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786207919384338-8288-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786207919384338-8288-0/minicluster-data/ts-0/wal
uuid: "440bde2bb1ba4f829a04dc0cac541751"
format_stamp: "Formatted at 2026-08-08 16:55:54 on dist-test-slave-6zbq"
server_key: "01b377a17de586a89334ebb71d545d05"
server_key_iv: "386dd0daadefa848104de7bc1bd45887"
server_key_version: "encryptionkey@0"
I20260808 16:55:54.888743 6360 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786207919384338-8288-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786207919384338-8288-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786207919384338-8288-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:55:54.914198 6360 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:55:54.916153 6360 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:55:54.916298 6360 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:55:54.916487 6360 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:55:54.916756 6360 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:55:54.916790 6360 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:55:54.916817 6360 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:55:54.916834 6360 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:55:54.922222 6360 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.1:38245
I20260808 16:55:54.922258 6559 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.1:38245 every 8 connection(s)
I20260808 16:55:54.922560 6360 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786207919384338-8288-0/minicluster-data/ts-0/data/info.pb
Aug 08 16:55:54 dist-test-slave-6zbq krb5kdc[6197](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208154, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.1@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260808 16:55:54.929152 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 6360
I20260808 16:55:54.929260 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786207919384338-8288-0/minicluster-data/ts-0/wal/instance
I20260808 16:55:54.929461 8288 external_mini_cluster.cc:1428] Setting key 2b995d8b57cfac82b91ec19d377e772f
I20260808 16:55:54.930748 6320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:54.923183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:44565 (local address 127.8.24.62:42089)
0808 16:55:54.923259 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0808 16:55:54.923262 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 16:55:54.923384 (+ 122us) server_negotiation.cc:407] Connection header received
0808 16:55:54.923806 (+ 422us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:55:54.923807 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:55:54.923816 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:55:54.923866 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:55:54.924666 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:54.925145 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:54.926050 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:54.926259 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:54.926319 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:54.928119 (+ 1800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:55:54.928121 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:55:54.928121 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:55:54.928128 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:55:54.929341 (+ 1213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:55:54.929737 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:55:54.929738 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:55:54.929738 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:55:54.929805 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:55:54.930064 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:55:54.930066 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:55:54.930066 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:55:54.930212 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:55:54.930285 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 16:55:54.930565 (+ 280us) server_negotiation.cc:299] Negotiation successful
0808 16:55:54.930661 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 16:55:54.930791 6565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:54.923152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:42089 (local address 127.8.24.1:44565)
0808 16:55:54.923282 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:54.923314 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0808 16:55:54.923735 (+ 421us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:55:54.923915 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:55:54.923918 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:55:54.924429 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:55:54.924601 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:54.924605 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:54.925212 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:54.925213 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:54.925974 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:54.925976 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:54.926327 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:54.926328 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:54.926461 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:54.926824 (+ 363us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:55:54.926831 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:55:54.928033 (+ 1202us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:55:54.929423 (+ 1390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:55:54.929431 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:55:54.929433 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:55:54.929681 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:55:54.929875 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:55:54.929876 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:55:54.929877 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:55:54.930000 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:55:54.930278 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:55:54.930293 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:55:54.930444 (+ 151us) client_negotiation.cc:769] Sending connection context
0808 16:55:54.930529 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 16:55:54.930644 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":69,"threads_started":1}
I20260808 16:55:54.931190 6561 heartbeater.cc:347] Connected to a master server at 127.8.24.62:42089
I20260808 16:55:54.931274 6561 heartbeater.cc:464] Registering TS with master...
I20260808 16:55:54.931443 6561 heartbeater.cc:511] Master 127.8.24.62:42089 requested a full tablet report, sending...
I20260808 16:55:54.931962 6271 ts_manager.cc:194] Registered new tserver with Master: 440bde2bb1ba4f829a04dc0cac541751 (127.8.24.1:38245)
I20260808 16:55:54.932531 6271 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.1@KRBTEST.COM'} at 127.8.24.1:44565
WARNING: no policy specified for kudu/127.8.24.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.2.keytab.
Entry for principal kudu/127.8.24.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.2.keytab.
I20260808 16:55:54.951247 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786207919384338-8288-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786207919384338-8288-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786207919384338-8288-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786207919384338-8288-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.2:0
--local_ip_for_outbound_sockets=127.8.24.2
--webserver_interface=127.8.24.2
--webserver_port=0
--tserver_master_addrs=127.8.24.62:42089
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.2.keytab
--principal=kudu/127.8.24.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:44977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:55:55.029027 6569 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:55:55.029211 6569 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:55:55.029234 6569 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:55:55.030277 6569 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:55:55.030308 6569 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:55:55.030331 6569 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.2
I20260808 16:55:55.031710 6569 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:44977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786207919384338-8288-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786207919384338-8288-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.2.keytab
--principal=kudu/127.8.24.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786207919384338-8288-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.24.2
--webserver_port=0
--tserver_master_addrs=127.8.24.62:42089
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6569
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.2
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786207919384338-8288-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:55:55.031877 6569 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:55:55.032038 6569 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:55:55.036641 6579 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:55:55.036670 6585 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:55:55.036687 6578 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:55:55.036824 6569 server_base.cc:1034] running on GCE node
I20260808 16:55:55.036994 6569 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:55:55.037178 6569 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:55:55.038306 6569 hybrid_clock.cc:648] HybridClock initialized: now 1786208155038297 us; error 26 us; skew 500 ppm
Aug 08 16:55:55 dist-test-slave-6zbq krb5kdc[6197](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208155, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:55:55.040518 6569 init.cc:377] Logged in from keytab as kudu/127.8.24.2@KRBTEST.COM (short username kudu)
I20260808 16:55:55.040896 6569 webserver.cc:460] Webserver started at http://127.8.24.2:38143/ using document root <none> and password file <none>
I20260808 16:55:55.041077 6569 fs_manager.cc:359] Metadata directory not provided
I20260808 16:55:55.041136 6569 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:55:55.041260 6569 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:55:55.042212 6569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786207919384338-8288-0/minicluster-data/ts-1/data/instance:
uuid: "7ece7c105ef641b2bff76b44a2244f04"
format_stamp: "Formatted at 2026-08-08 16:55:55 on dist-test-slave-6zbq"
server_key: "75b47859a590d6851fe65143fb58f028"
server_key_iv: "bd22e4839debb7a382b0c1b759faf483"
server_key_version: "encryptionkey@0"
I20260808 16:55:55.042568 6569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786207919384338-8288-0/minicluster-data/ts-1/wal/instance:
uuid: "7ece7c105ef641b2bff76b44a2244f04"
format_stamp: "Formatted at 2026-08-08 16:55:55 on dist-test-slave-6zbq"
server_key: "75b47859a590d6851fe65143fb58f028"
server_key_iv: "bd22e4839debb7a382b0c1b759faf483"
server_key_version: "encryptionkey@0"
I20260808 16:55:55.043763 6569 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 16:55:55.044410 6597 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:55:55.044559 6569 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 16:55:55.044639 6569 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786207919384338-8288-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786207919384338-8288-0/minicluster-data/ts-1/wal
uuid: "7ece7c105ef641b2bff76b44a2244f04"
format_stamp: "Formatted at 2026-08-08 16:55:55 on dist-test-slave-6zbq"
server_key: "75b47859a590d6851fe65143fb58f028"
server_key_iv: "bd22e4839debb7a382b0c1b759faf483"
server_key_version: "encryptionkey@0"
I20260808 16:55:55.044697 6569 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786207919384338-8288-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786207919384338-8288-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786207919384338-8288-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:55:55.065501 6569 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:55:55.067678 6569 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:55:55.067800 6569 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:55:55.067986 6569 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:55:55.068248 6569 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:55:55.068289 6569 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:55:55.068317 6569 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:55:55.068341 6569 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:55:55.073552 6569 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.2:40063
I20260808 16:55:55.073611 6768 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.2:40063 every 8 connection(s)
I20260808 16:55:55.073853 6569 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786207919384338-8288-0/minicluster-data/ts-1/data/info.pb
I20260808 16:55:55.075532 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 6569
I20260808 16:55:55.075623 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786207919384338-8288-0/minicluster-data/ts-1/wal/instance
I20260808 16:55:55.075803 8288 external_mini_cluster.cc:1428] Setting key 5f9e52738fbafcaf35cc7b69d172da02
Aug 08 16:55:55 dist-test-slave-6zbq krb5kdc[6197](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208155, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.2@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260808 16:55:55.082583 6320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:55.074455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:50943 (local address 127.8.24.62:42089)
0808 16:55:55.074509 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 16:55:55.074512 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 16:55:55.074682 (+ 170us) server_negotiation.cc:407] Connection header received
0808 16:55:55.075093 (+ 411us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:55:55.075094 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:55:55.075106 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:55:55.075154 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:55:55.076127 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:55.076755 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:55.077801 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:55.078076 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:55.078138 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:55.080076 (+ 1938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:55:55.080077 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:55:55.080077 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:55:55.080085 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:55:55.081358 (+ 1273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:55:55.081701 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:55:55.081702 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:55:55.081703 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:55:55.081761 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:55:55.081978 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:55:55.081979 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:55:55.081979 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:55:55.082086 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:55:55.082140 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 16:55:55.082407 (+ 267us) server_negotiation.cc:299] Negotiation successful
0808 16:55:55.082497 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 16:55:55.082715 6773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:55.074451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:42089 (local address 127.8.24.2:50943)
0808 16:55:55.074588 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:55.074620 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0808 16:55:55.075022 (+ 402us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:55:55.075199 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:55:55.075202 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:55:55.075758 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:55:55.075990 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:55.075996 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:55.076864 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:55.076865 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:55.077709 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:55.077712 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:55.078158 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:55.078159 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:55.078290 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:55.078754 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:55:55.078764 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:55:55.079992 (+ 1228us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:55:55.081429 (+ 1437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:55:55.081435 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:55:55.081437 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:55:55.081636 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:55:55.081824 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:55:55.081825 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:55:55.081825 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:55:55.081924 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:55:55.082140 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:55:55.082146 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:55:55.082275 (+ 129us) client_negotiation.cc:769] Sending connection context
0808 16:55:55.082379 (+ 104us) client_negotiation.cc:240] Negotiation successful
0808 16:55:55.082521 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":73,"threads_started":1}
I20260808 16:55:55.083197 6770 heartbeater.cc:347] Connected to a master server at 127.8.24.62:42089
I20260808 16:55:55.083307 6770 heartbeater.cc:464] Registering TS with master...
I20260808 16:55:55.083487 6770 heartbeater.cc:511] Master 127.8.24.62:42089 requested a full tablet report, sending...
I20260808 16:55:55.083868 6271 ts_manager.cc:194] Registered new tserver with Master: 7ece7c105ef641b2bff76b44a2244f04 (127.8.24.2:40063)
WARNING: no policy specified for kudu/127.8.24.3@KRBTEST.COM; defaulting to no policy
I20260808 16:55:55.084246 6271 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.2@KRBTEST.COM'} at 127.8.24.2:50943
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.3.keytab.
Entry for principal kudu/127.8.24.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.3.keytab.
I20260808 16:55:55.098459 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786207919384338-8288-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786207919384338-8288-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786207919384338-8288-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786207919384338-8288-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.3:0
--local_ip_for_outbound_sockets=127.8.24.3
--webserver_interface=127.8.24.3
--webserver_port=0
--tserver_master_addrs=127.8.24.62:42089
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.3.keytab
--principal=kudu/127.8.24.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:44977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:55:55.176451 6777 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:55:55.176631 6777 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:55:55.176654 6777 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:55:55.177734 6777 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:55:55.177772 6777 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:55:55.177802 6777 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.3
I20260808 16:55:55.179216 6777 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:44977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786207919384338-8288-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786207919384338-8288-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.3.keytab
--principal=kudu/127.8.24.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786207919384338-8288-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.24.3
--webserver_port=0
--tserver_master_addrs=127.8.24.62:42089
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6777
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.3
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786207919384338-8288-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:55:55.179411 6777 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:55:55.179597 6777 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:55:55.184039 6791 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:55:55.184094 6785 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:55:55.184060 6789 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:55:55.184206 6777 server_base.cc:1034] running on GCE node
I20260808 16:55:55.184388 6777 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:55:55.184607 6777 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:55:55.185734 6777 hybrid_clock.cc:648] HybridClock initialized: now 1786208155185722 us; error 27 us; skew 500 ppm
Aug 08 16:55:55 dist-test-slave-6zbq krb5kdc[6197](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208155, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:55:55.187938 6777 init.cc:377] Logged in from keytab as kudu/127.8.24.3@KRBTEST.COM (short username kudu)
I20260808 16:55:55.188342 6777 webserver.cc:460] Webserver started at http://127.8.24.3:37141/ using document root <none> and password file <none>
I20260808 16:55:55.188526 6777 fs_manager.cc:359] Metadata directory not provided
I20260808 16:55:55.188570 6777 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:55:55.188686 6777 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:55:55.189594 6777 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786207919384338-8288-0/minicluster-data/ts-2/data/instance:
uuid: "dc6dbad309b144dea57e47585c127fc8"
format_stamp: "Formatted at 2026-08-08 16:55:55 on dist-test-slave-6zbq"
server_key: "14fa2952d7bf099640f92ccdee2ec4e2"
server_key_iv: "dd890bb6aba83a56e56ad28f9b26f0e9"
server_key_version: "encryptionkey@0"
I20260808 16:55:55.189908 6777 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786207919384338-8288-0/minicluster-data/ts-2/wal/instance:
uuid: "dc6dbad309b144dea57e47585c127fc8"
format_stamp: "Formatted at 2026-08-08 16:55:55 on dist-test-slave-6zbq"
server_key: "14fa2952d7bf099640f92ccdee2ec4e2"
server_key_iv: "dd890bb6aba83a56e56ad28f9b26f0e9"
server_key_version: "encryptionkey@0"
I20260808 16:55:55.191126 6777 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 16:55:55.191781 6800 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:55:55.191941 6777 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 16:55:55.192024 6777 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786207919384338-8288-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786207919384338-8288-0/minicluster-data/ts-2/wal
uuid: "dc6dbad309b144dea57e47585c127fc8"
format_stamp: "Formatted at 2026-08-08 16:55:55 on dist-test-slave-6zbq"
server_key: "14fa2952d7bf099640f92ccdee2ec4e2"
server_key_iv: "dd890bb6aba83a56e56ad28f9b26f0e9"
server_key_version: "encryptionkey@0"
I20260808 16:55:55.192094 6777 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786207919384338-8288-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786207919384338-8288-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786207919384338-8288-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:55:55.202512 6777 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:55:55.203982 6777 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:55:55.204103 6777 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:55:55.204293 6777 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:55:55.204546 6777 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:55:55.204578 6777 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:55:55.204604 6777 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:55:55.204627 6777 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:55:55.209678 6777 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.3:45719
I20260808 16:55:55.209736 6970 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.3:45719 every 8 connection(s)
I20260808 16:55:55.209985 6777 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786207919384338-8288-0/minicluster-data/ts-2/data/info.pb
I20260808 16:55:55.212385 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 6777
I20260808 16:55:55.212479 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786207919384338-8288-0/minicluster-data/ts-2/wal/instance
I20260808 16:55:55.212646 8288 external_mini_cluster.cc:1428] Setting key 3ed00378fd9523bc6ad306e7c404eec8
Aug 08 16:55:55 dist-test-slave-6zbq krb5kdc[6197](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208155, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.3@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260808 16:55:55.217999 6320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:55.210563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:46235 (local address 127.8.24.62:42089)
0808 16:55:55.210611 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 16:55:55.210613 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:55:55.210797 (+ 184us) server_negotiation.cc:407] Connection header received
0808 16:55:55.211203 (+ 406us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:55:55.211204 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:55:55.211213 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:55:55.211252 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:55:55.212052 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:55.212592 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:55.213753 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:55.213959 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:55.214014 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:55.215812 (+ 1798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:55:55.215813 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:55:55.215813 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:55:55.215820 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:55:55.216842 (+ 1022us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:55:55.217173 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:55:55.217174 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:55:55.217174 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:55:55.217238 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:55:55.217439 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:55:55.217440 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:55:55.217441 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:55:55.217544 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:55:55.217597 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 16:55:55.217837 (+ 240us) server_negotiation.cc:299] Negotiation successful
0808 16:55:55.217912 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 16:55:55.218127 6973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:55.210542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:42089 (local address 127.8.24.3:46235)
0808 16:55:55.210694 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:55.210725 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0808 16:55:55.211112 (+ 387us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:55:55.211311 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:55:55.211314 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:55:55.211823 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:55:55.211987 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:55.211992 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:55.212680 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:55.212681 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:55.213674 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:55.213677 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:55.214028 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:55.214029 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:55.214146 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:55.214544 (+ 398us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:55:55.214551 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:55:55.215731 (+ 1180us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:55:55.216909 (+ 1178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:55:55.216915 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:55:55.216917 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:55:55.217110 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:55:55.217290 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:55:55.217291 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:55:55.217291 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:55:55.217386 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:55:55.217595 (+ 209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:55:55.217601 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:55:55.217727 (+ 126us) client_negotiation.cc:769] Sending connection context
0808 16:55:55.217808 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 16:55:55.217945 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":79,"threads_started":1}
I20260808 16:55:55.218565 6971 heartbeater.cc:347] Connected to a master server at 127.8.24.62:42089
I20260808 16:55:55.218649 6971 heartbeater.cc:464] Registering TS with master...
I20260808 16:55:55.218811 6971 heartbeater.cc:511] Master 127.8.24.62:42089 requested a full tablet report, sending...
I20260808 16:55:55.219137 6271 ts_manager.cc:194] Registered new tserver with Master: dc6dbad309b144dea57e47585c127fc8 (127.8.24.3:45719)
I20260808 16:55:55.219478 6271 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.3@KRBTEST.COM'} at 127.8.24.3:46235
I20260808 16:55:55.224123 8288 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 16:55:55.230509 6320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:55.225326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35540 (local address 127.8.24.62:42089)
0808 16:55:55.225360 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0808 16:55:55.225361 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 16:55:55.225470 (+ 109us) server_negotiation.cc:407] Connection header received
0808 16:55:55.225527 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:55:55.225528 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:55:55.225536 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:55:55.225582 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:55:55.226085 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:55.226570 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:55.227278 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:55.227451 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:55.227504 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:55.228511 (+ 1007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:55:55.228513 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:55:55.228513 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:55:55.228519 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:55:55.229486 (+ 967us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:55:55.229773 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:55:55.229774 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:55:55.229774 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:55:55.229829 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:55:55.229994 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:55:55.229995 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:55:55.229995 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:55:55.230120 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:55:55.230208 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 16:55:55.230365 (+ 157us) server_negotiation.cc:299] Negotiation successful
0808 16:55:55.230430 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260808 16:55:55.231843 6271 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35540:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 16:55:55.233450 6271 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 16:55:55.239202 6987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:55.235535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42416 (local address 127.8.24.3:45719)
0808 16:55:55.235664 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0808 16:55:55.235668 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 16:55:55.235678 (+ 10us) server_negotiation.cc:407] Connection header received
0808 16:55:55.235723 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:55:55.235725 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:55:55.235739 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:55:55.235821 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:55:55.235992 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:55.236751 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:55.238010 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:55.238819 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:55.238877 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:55.238929 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 16:55:55.239088 (+ 159us) server_negotiation.cc:299] Negotiation successful
0808 16:55:55.239127 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":56,"threads_started":1}
I20260808 16:55:55.239253 6984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:55.235339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.2:40063 (local address 127.0.0.1:59262)
0808 16:55:55.235445 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:55.235481 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 16:55:55.235543 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:55:55.235914 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:55:55.235915 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:55:55.235917 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:55:55.235960 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:55.235961 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:55.237194 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:55.237195 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:55.238171 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:55.238173 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:55.239014 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:55.239015 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:55.239069 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:55.239072 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 16:55:55.239134 (+ 62us) client_negotiation.cc:240] Negotiation successful
0808 16:55:55.239191 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":83,"thread_start_us":63,"threads_started":1}
I20260808 16:55:55.239413 6986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:55.235342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.3:45719 (local address 127.0.0.1:42416)
0808 16:55:55.235490 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:55.235500 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 16:55:55.235543 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:55:55.235832 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:55:55.235835 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:55:55.235837 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:55:55.235897 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:55.235898 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:55.236825 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:55.236826 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:55.237927 (+ 1101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:55.237929 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:55.238891 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:55.238892 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:55.238972 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:55.238986 (+ 14us) client_negotiation.cc:769] Sending connection context
0808 16:55:55.239313 (+ 327us) client_negotiation.cc:240] Negotiation successful
0808 16:55:55.239354 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":70,"threads_started":1}
I20260808 16:55:55.239567 6990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:55.235596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59262 (local address 127.8.24.2:40063)
0808 16:55:55.235758 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0808 16:55:55.235762 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 16:55:55.235770 (+ 8us) server_negotiation.cc:407] Connection header received
0808 16:55:55.235799 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:55:55.235801 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:55:55.235816 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:55:55.236025 (+ 209us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:55:55.236093 (+ 68us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:55.237087 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:55.238242 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:55.238910 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:55.239013 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:55.239067 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 16:55:55.239149 (+ 82us) server_negotiation.cc:299] Negotiation successful
0808 16:55:55.239194 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":88,"threads_started":1}
I20260808 16:55:55.239647 6982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:55.235342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:38245 (local address 127.0.0.1:58720)
0808 16:55:55.235972 (+ 630us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:55.235987 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 16:55:55.236046 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:55:55.236511 (+ 465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:55:55.236512 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:55:55.236514 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:55:55.236558 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:55.236559 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:55.237566 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:55.237567 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:55.238440 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:55.238443 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:55.239364 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:55.239365 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:55.239425 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:55.239429 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 16:55:55.239546 (+ 117us) client_negotiation.cc:240] Negotiation successful
0808 16:55:55.239585 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":618,"mutex_wait_us":2,"thread_start_us":53,"threads_started":1}
I20260808 16:55:55.239872 6992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:55.235797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58720 (local address 127.8.24.1:38245)
0808 16:55:55.235943 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0808 16:55:55.235947 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 16:55:55.236048 (+ 101us) server_negotiation.cc:407] Connection header received
0808 16:55:55.236125 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:55:55.236128 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:55:55.236143 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:55:55.236508 (+ 365us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:55:55.236618 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:55.237484 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:55.238518 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:55.239209 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:55.239271 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:55.239328 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 16:55:55.239753 (+ 425us) server_negotiation.cc:299] Negotiation successful
0808 16:55:55.239795 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":82,"threads_started":1}
I20260808 16:55:55.240051 6874 tablet_service.cc:1467] Processing CreateTablet for tablet 2e2f31637ec844b89951ee1c9b8b6753 (DEFAULT_TABLE table=test-table [id=b53eb75756744874b19cc41257163859]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:55:55.240170 6670 tablet_service.cc:1467] Processing CreateTablet for tablet 2e2f31637ec844b89951ee1c9b8b6753 (DEFAULT_TABLE table=test-table [id=b53eb75756744874b19cc41257163859]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:55:55.240350 6874 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e2f31637ec844b89951ee1c9b8b6753. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:55:55.240486 6670 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e2f31637ec844b89951ee1c9b8b6753. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:55:55.242801 6994 tablet_bootstrap.cc:492] T 2e2f31637ec844b89951ee1c9b8b6753 P dc6dbad309b144dea57e47585c127fc8: Bootstrap starting.
I20260808 16:55:55.242959 6463 tablet_service.cc:1467] Processing CreateTablet for tablet 2e2f31637ec844b89951ee1c9b8b6753 (DEFAULT_TABLE table=test-table [id=b53eb75756744874b19cc41257163859]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:55:55.243173 6995 tablet_bootstrap.cc:492] T 2e2f31637ec844b89951ee1c9b8b6753 P 7ece7c105ef641b2bff76b44a2244f04: Bootstrap starting.
I20260808 16:55:55.243279 6463 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e2f31637ec844b89951ee1c9b8b6753. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:55:55.243547 6994 tablet_bootstrap.cc:654] T 2e2f31637ec844b89951ee1c9b8b6753 P dc6dbad309b144dea57e47585c127fc8: Neither blocks nor log segments found. Creating new log.
I20260808 16:55:55.243757 6995 tablet_bootstrap.cc:654] T 2e2f31637ec844b89951ee1c9b8b6753 P 7ece7c105ef641b2bff76b44a2244f04: Neither blocks nor log segments found. Creating new log.
I20260808 16:55:55.244035 6994 log.cc:824] T 2e2f31637ec844b89951ee1c9b8b6753 P dc6dbad309b144dea57e47585c127fc8: Log is configured to *not* fsync() on all Append() calls
I20260808 16:55:55.244263 6995 log.cc:824] T 2e2f31637ec844b89951ee1c9b8b6753 P 7ece7c105ef641b2bff76b44a2244f04: Log is configured to *not* fsync() on all Append() calls
I20260808 16:55:55.244750 6994 tablet_bootstrap.cc:492] T 2e2f31637ec844b89951ee1c9b8b6753 P dc6dbad309b144dea57e47585c127fc8: No bootstrap required, opened a new log
I20260808 16:55:55.244812 6994 ts_tablet_manager.cc:1397] T 2e2f31637ec844b89951ee1c9b8b6753 P dc6dbad309b144dea57e47585c127fc8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 16:55:55.244983 6995 tablet_bootstrap.cc:492] T 2e2f31637ec844b89951ee1c9b8b6753 P 7ece7c105ef641b2bff76b44a2244f04: No bootstrap required, opened a new log
I20260808 16:55:55.245055 6995 ts_tablet_manager.cc:1397] T 2e2f31637ec844b89951ee1c9b8b6753 P 7ece7c105ef641b2bff76b44a2244f04: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 16:55:55.246300 7001 tablet_bootstrap.cc:492] T 2e2f31637ec844b89951ee1c9b8b6753 P 440bde2bb1ba4f829a04dc0cac541751: Bootstrap starting.
I20260808 16:55:55.246117 6994 raft_consensus.cc:348] T 2e2f31637ec844b89951ee1c9b8b6753 P dc6dbad309b144dea57e47585c127fc8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "440bde2bb1ba4f829a04dc0cac541751" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 38245 } } peers { permanent_uuid: "7ece7c105ef641b2bff76b44a2244f04" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 40063 } } peers { permanent_uuid: "dc6dbad309b144dea57e47585c127fc8" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 45719 } }
I20260808 16:55:55.246411 6994 raft_consensus.cc:374] T 2e2f31637ec844b89951ee1c9b8b6753 P dc6dbad309b144dea57e47585c127fc8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:55:55.246462 6994 raft_consensus.cc:729] T 2e2f31637ec844b89951ee1c9b8b6753 P dc6dbad309b144dea57e47585c127fc8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc6dbad309b144dea57e47585c127fc8, State: Initialized, Role: FOLLOWER
I20260808 16:55:55.246640 6994 consensus_queue.cc:260] T 2e2f31637ec844b89951ee1c9b8b6753 P dc6dbad309b144dea57e47585c127fc8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "440bde2bb1ba4f829a04dc0cac541751" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 38245 } } peers { permanent_uuid: "7ece7c105ef641b2bff76b44a2244f04" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 40063 } } peers { permanent_uuid: "dc6dbad309b144dea57e47585c127fc8" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 45719 } }
I20260808 16:55:55.246950 6994 ts_tablet_manager.cc:1428] T 2e2f31637ec844b89951ee1c9b8b6753 P dc6dbad309b144dea57e47585c127fc8: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 16:55:55.246933 6995 raft_consensus.cc:348] T 2e2f31637ec844b89951ee1c9b8b6753 P 7ece7c105ef641b2bff76b44a2244f04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "440bde2bb1ba4f829a04dc0cac541751" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 38245 } } peers { permanent_uuid: "7ece7c105ef641b2bff76b44a2244f04" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 40063 } } peers { permanent_uuid: "dc6dbad309b144dea57e47585c127fc8" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 45719 } }
I20260808 16:55:55.247102 6995 raft_consensus.cc:374] T 2e2f31637ec844b89951ee1c9b8b6753 P 7ece7c105ef641b2bff76b44a2244f04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:55:55.247115 6971 heartbeater.cc:503] Master 127.8.24.62:42089 was elected leader, sending a full tablet report...
I20260808 16:55:55.247133 7001 tablet_bootstrap.cc:654] T 2e2f31637ec844b89951ee1c9b8b6753 P 440bde2bb1ba4f829a04dc0cac541751: Neither blocks nor log segments found. Creating new log.
I20260808 16:55:55.247145 6995 raft_consensus.cc:729] T 2e2f31637ec844b89951ee1c9b8b6753 P 7ece7c105ef641b2bff76b44a2244f04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ece7c105ef641b2bff76b44a2244f04, State: Initialized, Role: FOLLOWER
I20260808 16:55:55.247256 6995 consensus_queue.cc:260] T 2e2f31637ec844b89951ee1c9b8b6753 P 7ece7c105ef641b2bff76b44a2244f04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "440bde2bb1ba4f829a04dc0cac541751" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 38245 } } peers { permanent_uuid: "7ece7c105ef641b2bff76b44a2244f04" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 40063 } } peers { permanent_uuid: "dc6dbad309b144dea57e47585c127fc8" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 45719 } }
I20260808 16:55:55.247506 6995 ts_tablet_manager.cc:1428] T 2e2f31637ec844b89951ee1c9b8b6753 P 7ece7c105ef641b2bff76b44a2244f04: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260808 16:55:55.247532 6770 heartbeater.cc:503] Master 127.8.24.62:42089 was elected leader, sending a full tablet report...
I20260808 16:55:55.247656 7001 log.cc:824] T 2e2f31637ec844b89951ee1c9b8b6753 P 440bde2bb1ba4f829a04dc0cac541751: Log is configured to *not* fsync() on all Append() calls
I20260808 16:55:55.248474 7001 tablet_bootstrap.cc:492] T 2e2f31637ec844b89951ee1c9b8b6753 P 440bde2bb1ba4f829a04dc0cac541751: No bootstrap required, opened a new log
I20260808 16:55:55.248543 7001 ts_tablet_manager.cc:1397] T 2e2f31637ec844b89951ee1c9b8b6753 P 440bde2bb1ba4f829a04dc0cac541751: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 16:55:55.249782 7001 raft_consensus.cc:348] T 2e2f31637ec844b89951ee1c9b8b6753 P 440bde2bb1ba4f829a04dc0cac541751 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "440bde2bb1ba4f829a04dc0cac541751" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 38245 } } peers { permanent_uuid: "7ece7c105ef641b2bff76b44a2244f04" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 40063 } } peers { permanent_uuid: "dc6dbad309b144dea57e47585c127fc8" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 45719 } }
I20260808 16:55:55.249905 7001 raft_consensus.cc:374] T 2e2f31637ec844b89951ee1c9b8b6753 P 440bde2bb1ba4f829a04dc0cac541751 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:55:55.249926 7001 raft_consensus.cc:729] T 2e2f31637ec844b89951ee1c9b8b6753 P 440bde2bb1ba4f829a04dc0cac541751 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 440bde2bb1ba4f829a04dc0cac541751, State: Initialized, Role: FOLLOWER
I20260808 16:55:55.249997 7001 consensus_queue.cc:260] T 2e2f31637ec844b89951ee1c9b8b6753 P 440bde2bb1ba4f829a04dc0cac541751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "440bde2bb1ba4f829a04dc0cac541751" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 38245 } } peers { permanent_uuid: "7ece7c105ef641b2bff76b44a2244f04" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 40063 } } peers { permanent_uuid: "dc6dbad309b144dea57e47585c127fc8" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 45719 } }
I20260808 16:55:55.250178 7001 ts_tablet_manager.cc:1428] T 2e2f31637ec844b89951ee1c9b8b6753 P 440bde2bb1ba4f829a04dc0cac541751: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 16:55:55.250239 6561 heartbeater.cc:503] Master 127.8.24.62:42089 was elected leader, sending a full tablet report...
W20260808 16:55:55.324338 6772 tablet.cc:2369] T 2e2f31637ec844b89951ee1c9b8b6753 P 7ece7c105ef641b2bff76b44a2244f04: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 16:55:55.338395 7011 raft_consensus.cc:482] T 2e2f31637ec844b89951ee1c9b8b6753 P 440bde2bb1ba4f829a04dc0cac541751 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 16:55:55.338501 7011 raft_consensus.cc:504] T 2e2f31637ec844b89951ee1c9b8b6753 P 440bde2bb1ba4f829a04dc0cac541751 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "440bde2bb1ba4f829a04dc0cac541751" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 38245 } } peers { permanent_uuid: "7ece7c105ef641b2bff76b44a2244f04" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 40063 } } peers { permanent_uuid: "dc6dbad309b144dea57e47585c127fc8" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 45719 } }
I20260808 16:55:55.338814 7011 leader_election.cc:288] T 2e2f31637ec844b89951ee1c9b8b6753 P 440bde2bb1ba4f829a04dc0cac541751 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7ece7c105ef641b2bff76b44a2244f04 (127.8.24.2:40063), dc6dbad309b144dea57e47585c127fc8 (127.8.24.3:45719)
I20260808 16:55:55.341439 6987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:55.339049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:47065 (local address 127.8.24.3:45719)
0808 16:55:55.339089 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 16:55:55.339091 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:55:55.339097 (+ 6us) server_negotiation.cc:407] Connection header received
0808 16:55:55.339127 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:55:55.339128 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:55:55.339137 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:55:55.339193 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:55:55.339349 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:55.339814 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:55.340739 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:55.341126 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:55.341186 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:55.341198 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0808 16:55:55.341336 (+ 138us) server_negotiation.cc:299] Negotiation successful
0808 16:55:55.341375 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 16:55:55.341439 6565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:55.338962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.3:45719 (local address 127.8.24.1:47065)
0808 16:55:55.339003 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:55.339014 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 16:55:55.339052 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:55:55.339213 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:55:55.339215 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:55:55.339217 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:55:55.339284 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:55.339286 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:55.339890 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:55.339891 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:55.340648 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:55.340650 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:55.341202 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:55.341203 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:55.341272 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:55.341275 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 16:55:55.341321 (+ 46us) client_negotiation.cc:240] Negotiation successful
0808 16:55:55.341364 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21,"mutex_wait_us":28}
I20260808 16:55:55.341550 6990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:55.339040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:59415 (local address 127.8.24.2:40063)
0808 16:55:55.339084 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 16:55:55.339086 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:55:55.339240 (+ 154us) server_negotiation.cc:407] Connection header received
0808 16:55:55.339273 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:55:55.339274 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:55:55.339288 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:55:55.339336 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:55:55.339477 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:55.339939 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:55.340839 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:55.341240 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:55.341300 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:55.341310 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 16:55:55.341455 (+ 145us) server_negotiation.cc:299] Negotiation successful
0808 16:55:55.341491 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 16:55:55.341558 7012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:55.338962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.2:40063 (local address 127.8.24.1:59415)
0808 16:55:55.339173 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:55.339180 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0808 16:55:55.339225 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:55:55.339345 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:55:55.339345 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:55:55.339347 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:55:55.339418 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:55.339420 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:55.340013 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:55.340014 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:55.340756 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:55.340758 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:55.341319 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:55.341320 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:55.341393 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:55.341396 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 16:55:55.341439 (+ 43us) client_negotiation.cc:240] Negotiation successful
0808 16:55:55.341475 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"mutex_wait_us":41,"thread_start_us":117,"threads_started":1}
I20260808 16:55:55.341918 6704 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e2f31637ec844b89951ee1c9b8b6753" candidate_uuid: "440bde2bb1ba4f829a04dc0cac541751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ece7c105ef641b2bff76b44a2244f04" is_pre_election: true
I20260808 16:55:55.341918 6907 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e2f31637ec844b89951ee1c9b8b6753" candidate_uuid: "440bde2bb1ba4f829a04dc0cac541751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc6dbad309b144dea57e47585c127fc8" is_pre_election: true
I20260808 16:55:55.342067 6704 raft_consensus.cc:2454] T 2e2f31637ec844b89951ee1c9b8b6753 P 7ece7c105ef641b2bff76b44a2244f04 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 440bde2bb1ba4f829a04dc0cac541751 in term 0.
I20260808 16:55:55.342067 6907 raft_consensus.cc:2454] T 2e2f31637ec844b89951ee1c9b8b6753 P dc6dbad309b144dea57e47585c127fc8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 440bde2bb1ba4f829a04dc0cac541751 in term 0.
I20260808 16:55:55.342288 6393 leader_election.cc:302] T 2e2f31637ec844b89951ee1c9b8b6753 P 440bde2bb1ba4f829a04dc0cac541751 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 440bde2bb1ba4f829a04dc0cac541751, 7ece7c105ef641b2bff76b44a2244f04; no voters:
I20260808 16:55:55.342455 7011 raft_consensus.cc:2781] T 2e2f31637ec844b89951ee1c9b8b6753 P 440bde2bb1ba4f829a04dc0cac541751 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 16:55:55.342514 7011 raft_consensus.cc:482] T 2e2f31637ec844b89951ee1c9b8b6753 P 440bde2bb1ba4f829a04dc0cac541751 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 16:55:55.342530 7011 raft_consensus.cc:3037] T 2e2f31637ec844b89951ee1c9b8b6753 P 440bde2bb1ba4f829a04dc0cac541751 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:55:55.343222 7011 raft_consensus.cc:504] T 2e2f31637ec844b89951ee1c9b8b6753 P 440bde2bb1ba4f829a04dc0cac541751 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "440bde2bb1ba4f829a04dc0cac541751" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 38245 } } peers { permanent_uuid: "7ece7c105ef641b2bff76b44a2244f04" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 40063 } } peers { permanent_uuid: "dc6dbad309b144dea57e47585c127fc8" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 45719 } }
I20260808 16:55:55.343343 7011 leader_election.cc:288] T 2e2f31637ec844b89951ee1c9b8b6753 P 440bde2bb1ba4f829a04dc0cac541751 [CANDIDATE]: Term 1 election: Requested vote from peers 7ece7c105ef641b2bff76b44a2244f04 (127.8.24.2:40063), dc6dbad309b144dea57e47585c127fc8 (127.8.24.3:45719)
I20260808 16:55:55.343474 6907 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e2f31637ec844b89951ee1c9b8b6753" candidate_uuid: "440bde2bb1ba4f829a04dc0cac541751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc6dbad309b144dea57e47585c127fc8"
I20260808 16:55:55.343482 6704 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e2f31637ec844b89951ee1c9b8b6753" candidate_uuid: "440bde2bb1ba4f829a04dc0cac541751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ece7c105ef641b2bff76b44a2244f04"
I20260808 16:55:55.343551 6704 raft_consensus.cc:3037] T 2e2f31637ec844b89951ee1c9b8b6753 P 7ece7c105ef641b2bff76b44a2244f04 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:55:55.343551 6907 raft_consensus.cc:3037] T 2e2f31637ec844b89951ee1c9b8b6753 P dc6dbad309b144dea57e47585c127fc8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:55:55.344218 6704 raft_consensus.cc:2454] T 2e2f31637ec844b89951ee1c9b8b6753 P 7ece7c105ef641b2bff76b44a2244f04 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 440bde2bb1ba4f829a04dc0cac541751 in term 1.
I20260808 16:55:55.344225 6907 raft_consensus.cc:2454] T 2e2f31637ec844b89951ee1c9b8b6753 P dc6dbad309b144dea57e47585c127fc8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 440bde2bb1ba4f829a04dc0cac541751 in term 1.
I20260808 16:55:55.344379 6393 leader_election.cc:302] T 2e2f31637ec844b89951ee1c9b8b6753 P 440bde2bb1ba4f829a04dc0cac541751 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 440bde2bb1ba4f829a04dc0cac541751, 7ece7c105ef641b2bff76b44a2244f04; no voters:
I20260808 16:55:55.344486 7011 raft_consensus.cc:2781] T 2e2f31637ec844b89951ee1c9b8b6753 P 440bde2bb1ba4f829a04dc0cac541751 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:55:55.344625 7011 raft_consensus.cc:686] T 2e2f31637ec844b89951ee1c9b8b6753 P 440bde2bb1ba4f829a04dc0cac541751 [term 1 LEADER]: Becoming Leader. State: Replica: 440bde2bb1ba4f829a04dc0cac541751, State: Running, Role: LEADER
I20260808 16:55:55.344722 7011 consensus_queue.cc:237] T 2e2f31637ec844b89951ee1c9b8b6753 P 440bde2bb1ba4f829a04dc0cac541751 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "440bde2bb1ba4f829a04dc0cac541751" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 38245 } } peers { permanent_uuid: "7ece7c105ef641b2bff76b44a2244f04" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 40063 } } peers { permanent_uuid: "dc6dbad309b144dea57e47585c127fc8" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 45719 } }
I20260808 16:55:55.345533 6271 catalog_manager.cc:5495] T 2e2f31637ec844b89951ee1c9b8b6753 P 440bde2bb1ba4f829a04dc0cac541751 reported cstate change: term changed from 0 to 1, leader changed from <none> to 440bde2bb1ba4f829a04dc0cac541751 (127.8.24.1). New cstate: current_term: 1 leader_uuid: "440bde2bb1ba4f829a04dc0cac541751" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "440bde2bb1ba4f829a04dc0cac541751" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 38245 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7ece7c105ef641b2bff76b44a2244f04" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 40063 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dc6dbad309b144dea57e47585c127fc8" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 45719 } health_report { overall_health: UNKNOWN } } }
I20260808 16:55:55.352207 6992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:55.350044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58722 (local address 127.8.24.1:38245)
0808 16:55:55.350086 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 16:55:55.350088 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:55:55.350108 (+ 20us) server_negotiation.cc:407] Connection header received
0808 16:55:55.350198 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:55:55.350199 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:55:55.350216 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:55:55.350269 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 16:55:55.350395 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:55.350812 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:55.351509 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:55.351731 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:55.351782 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:55.351926 (+ 144us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 16:55:55.352012 (+ 86us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 16:55:55.352078 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 16:55:55.352108 (+ 30us) server_negotiation.cc:299] Negotiation successful
0808 16:55:55.352123 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 16:55:55.353653 6907 raft_consensus.cc:1261] T 2e2f31637ec844b89951ee1c9b8b6753 P dc6dbad309b144dea57e47585c127fc8 [term 1 FOLLOWER]: Refusing update from remote peer 440bde2bb1ba4f829a04dc0cac541751: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:55:55.353677 6704 raft_consensus.cc:1261] T 2e2f31637ec844b89951ee1c9b8b6753 P 7ece7c105ef641b2bff76b44a2244f04 [term 1 FOLLOWER]: Refusing update from remote peer 440bde2bb1ba4f829a04dc0cac541751: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:55:55.353941 7015 consensus_queue.cc:1035] T 2e2f31637ec844b89951ee1c9b8b6753 P 440bde2bb1ba4f829a04dc0cac541751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc6dbad309b144dea57e47585c127fc8" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 45719 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:55:55.354053 7011 consensus_queue.cc:1035] T 2e2f31637ec844b89951ee1c9b8b6753 P 440bde2bb1ba4f829a04dc0cac541751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ece7c105ef641b2bff76b44a2244f04" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 40063 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:55:55.356334 7019 mvcc.cc:205] Tried to move back new op lower bound from 7316308604326342656 to 7316308604292505600. Current Snapshot: MvccSnapshot[applied={T|T < 7316308604326342656}]
I20260808 16:55:55.356401 7020 mvcc.cc:205] Tried to move back new op lower bound from 7316308604326342656 to 7316308604292505600. Current Snapshot: MvccSnapshot[applied={T|T < 7316308604326342656}]
I20260808 16:55:55.358654 6270 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35540:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 16:55:55.358734 6270 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35540:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 16:55:55.359385 6270 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c43eb7ef565944e08406ef06c634326d: Sending DeleteTablet for 3 replicas of tablet 2e2f31637ec844b89951ee1c9b8b6753
I20260808 16:55:55.359692 6670 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e2f31637ec844b89951ee1c9b8b6753 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:55:55 UTC) from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:59262
I20260808 16:55:55.359906 7034 tablet_replica.cc:330] T 2e2f31637ec844b89951ee1c9b8b6753 P 7ece7c105ef641b2bff76b44a2244f04: stopping tablet replica
I20260808 16:55:55.359895 6463 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e2f31637ec844b89951ee1c9b8b6753 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:55:55 UTC) from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:58720
I20260808 16:55:55.359997 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 6360
I20260808 16:55:55.360013 7034 raft_consensus.cc:2229] T 2e2f31637ec844b89951ee1c9b8b6753 P 7ece7c105ef641b2bff76b44a2244f04 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 16:55:55.360100 7034 raft_consensus.cc:2258] T 2e2f31637ec844b89951ee1c9b8b6753 P 7ece7c105ef641b2bff76b44a2244f04 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:55:55.364259 7034 ts_tablet_manager.cc:1905] T 2e2f31637ec844b89951ee1c9b8b6753 P 7ece7c105ef641b2bff76b44a2244f04: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:55:55.365708 7034 ts_tablet_manager.cc:1918] T 2e2f31637ec844b89951ee1c9b8b6753 P 7ece7c105ef641b2bff76b44a2244f04: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 16:55:55.365756 7034 log.cc:1196] T 2e2f31637ec844b89951ee1c9b8b6753 P 7ece7c105ef641b2bff76b44a2244f04: Deleting WAL directory at /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786207919384338-8288-0/minicluster-data/ts-1/wal/wals/2e2f31637ec844b89951ee1c9b8b6753
I20260808 16:55:55.365933 7034 ts_tablet_manager.cc:1939] T 2e2f31637ec844b89951ee1c9b8b6753 P 7ece7c105ef641b2bff76b44a2244f04: Deleting consensus metadata
I20260808 16:55:55.369798 6249 catalog_manager.cc:4841] TS 7ece7c105ef641b2bff76b44a2244f04 (127.8.24.2:40063): tablet 2e2f31637ec844b89951ee1c9b8b6753 (table test-table [id=b53eb75756744874b19cc41257163859]) successfully deleted
I20260808 16:55:55.370033 6874 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e2f31637ec844b89951ee1c9b8b6753 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:55:55 UTC) from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:42416
I20260808 16:55:55.370436 7036 tablet_replica.cc:330] T 2e2f31637ec844b89951ee1c9b8b6753 P dc6dbad309b144dea57e47585c127fc8: stopping tablet replica
I20260808 16:55:55.370513 7036 raft_consensus.cc:2229] T 2e2f31637ec844b89951ee1c9b8b6753 P dc6dbad309b144dea57e47585c127fc8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 16:55:55.370607 7036 raft_consensus.cc:2258] T 2e2f31637ec844b89951ee1c9b8b6753 P dc6dbad309b144dea57e47585c127fc8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:55:55.370947 7036 ts_tablet_manager.cc:1905] T 2e2f31637ec844b89951ee1c9b8b6753 P dc6dbad309b144dea57e47585c127fc8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:55:55.372005 7036 ts_tablet_manager.cc:1918] T 2e2f31637ec844b89951ee1c9b8b6753 P dc6dbad309b144dea57e47585c127fc8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 16:55:55.372061 7036 log.cc:1196] T 2e2f31637ec844b89951ee1c9b8b6753 P dc6dbad309b144dea57e47585c127fc8: Deleting WAL directory at /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786207919384338-8288-0/minicluster-data/ts-2/wal/wals/2e2f31637ec844b89951ee1c9b8b6753
I20260808 16:55:55.372264 7036 ts_tablet_manager.cc:1939] T 2e2f31637ec844b89951ee1c9b8b6753 P dc6dbad309b144dea57e47585c127fc8: Deleting consensus metadata
I20260808 16:55:55.372612 6254 catalog_manager.cc:4841] TS dc6dbad309b144dea57e47585c127fc8 (127.8.24.3:45719): tablet 2e2f31637ec844b89951ee1c9b8b6753 (table test-table [id=b53eb75756744874b19cc41257163859]) successfully deleted
W20260808 16:55:55.374359 6253 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.24.1:38245 (error 108)
I20260808 16:55:55.374773 6982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:55.374669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:38245 (local address 127.0.0.1:58726)
0808 16:55:55.374717 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:55.374750 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.1:38245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260808 16:55:55.374856 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 6569
W20260808 16:55:55.374886 6253 catalog_manager.cc:4584] TS 440bde2bb1ba4f829a04dc0cac541751 (127.8.24.1:38245): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2e2f31637ec844b89951ee1c9b8b6753: Network error: Client connection negotiation failed: client connection to 127.8.24.1:38245: connect: Connection refused (error 111)
I20260808 16:55:55.382418 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 6777
I20260808 16:55:55.390318 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 6220
2026-08-08T16:55:55Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2001 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-08T16:55:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T16:55:55Z Disabled control of system clock
I20260808 16:55:55.425442 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786207919384338-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786207919384338-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786207919384338-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:42825
--webserver_interface=127.8.24.62
--webserver_port=0
--builtin_ntp_servers=127.8.24.20:44391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:42825
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 16:55:55.503722 7041 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:55:55.503918 7041 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:55:55.503947 7041 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:55:55.505023 7041 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 16:55:55.505064 7041 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:55:55.505081 7041 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:55:55.505097 7041 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 16:55:55.505113 7041 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 16:55:55.506441 7041 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:44391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786207919384338-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786207919384338-8288-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:42825
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:42825
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.24.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7041
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786207919384338-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:55:55.506670 7041 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:55:55.506914 7041 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:55:55.511332 7051 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:55:55.511359 7048 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:55:55.511370 7047 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:55:55.511648 7041 server_base.cc:1034] running on GCE node
I20260808 16:55:55.511870 7041 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:55:55.512130 7041 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:55:55.513284 7041 hybrid_clock.cc:648] HybridClock initialized: now 1786208155513266 us; error 35 us; skew 500 ppm
I20260808 16:55:55.514425 7041 webserver.cc:460] Webserver started at http://127.8.24.62:42047/ using document root <none> and password file <none>
I20260808 16:55:55.514643 7041 fs_manager.cc:359] Metadata directory not provided
I20260808 16:55:55.514688 7041 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:55:55.514808 7041 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:55:55.515801 7041 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786207919384338-8288-0/minicluster-data/master-0/data/instance:
uuid: "1bcf2fec752a48bfb6da61a1212977c7"
format_stamp: "Formatted at 2026-08-08 16:55:55 on dist-test-slave-6zbq"
server_key: "4783865141007576fcb7b6f1aac53f3f"
server_key_iv: "353c762939453fb3292b0cbfbc1969bd"
server_key_version: "encryptionkey@0"
I20260808 16:55:55.516114 7041 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786207919384338-8288-0/minicluster-data/master-0/wal/instance:
uuid: "1bcf2fec752a48bfb6da61a1212977c7"
format_stamp: "Formatted at 2026-08-08 16:55:55 on dist-test-slave-6zbq"
server_key: "4783865141007576fcb7b6f1aac53f3f"
server_key_iv: "353c762939453fb3292b0cbfbc1969bd"
server_key_version: "encryptionkey@0"
I20260808 16:55:55.517233 7041 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 16:55:55.517823 7059 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:55:55.517952 7041 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 16:55:55.518030 7041 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786207919384338-8288-0/minicluster-data/master-0/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786207919384338-8288-0/minicluster-data/master-0/wal
uuid: "1bcf2fec752a48bfb6da61a1212977c7"
format_stamp: "Formatted at 2026-08-08 16:55:55 on dist-test-slave-6zbq"
server_key: "4783865141007576fcb7b6f1aac53f3f"
server_key_iv: "353c762939453fb3292b0cbfbc1969bd"
server_key_version: "encryptionkey@0"
I20260808 16:55:55.518095 7041 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786207919384338-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786207919384338-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786207919384338-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:55:55.526922 7041 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:55:55.527163 7041 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:55:55.527288 7041 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:55:55.530887 7041 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.62:42825
I20260808 16:55:55.530920 7130 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.62:42825 every 8 connection(s)
I20260808 16:55:55.531201 7041 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
I20260808 16:55:55.532049 7131 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:55:55.534119 7131 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1bcf2fec752a48bfb6da61a1212977c7: Bootstrap starting.
I20260808 16:55:55.534701 7131 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1bcf2fec752a48bfb6da61a1212977c7: Neither blocks nor log segments found. Creating new log.
I20260808 16:55:55.534933 7131 log.cc:824] T 00000000000000000000000000000000 P 1bcf2fec752a48bfb6da61a1212977c7: Log is configured to *not* fsync() on all Append() calls
I20260808 16:55:55.535504 7131 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1bcf2fec752a48bfb6da61a1212977c7: No bootstrap required, opened a new log
I20260808 16:55:55.536973 7131 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1bcf2fec752a48bfb6da61a1212977c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1bcf2fec752a48bfb6da61a1212977c7" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 42825 } }
I20260808 16:55:55.537098 7131 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1bcf2fec752a48bfb6da61a1212977c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:55:55.537127 7131 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1bcf2fec752a48bfb6da61a1212977c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bcf2fec752a48bfb6da61a1212977c7, State: Initialized, Role: FOLLOWER
I20260808 16:55:55.537235 7131 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1bcf2fec752a48bfb6da61a1212977c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1bcf2fec752a48bfb6da61a1212977c7" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 42825 } }
I20260808 16:55:55.537285 7131 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1bcf2fec752a48bfb6da61a1212977c7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 16:55:55.537305 7131 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1bcf2fec752a48bfb6da61a1212977c7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 16:55:55.537334 7131 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1bcf2fec752a48bfb6da61a1212977c7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:55:55.537762 7131 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1bcf2fec752a48bfb6da61a1212977c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1bcf2fec752a48bfb6da61a1212977c7" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 42825 } }
I20260808 16:55:55.537844 7131 leader_election.cc:302] T 00000000000000000000000000000000 P 1bcf2fec752a48bfb6da61a1212977c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1bcf2fec752a48bfb6da61a1212977c7; no voters:
I20260808 16:55:55.537959 7131 leader_election.cc:288] T 00000000000000000000000000000000 P 1bcf2fec752a48bfb6da61a1212977c7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 16:55:55.538013 7135 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1bcf2fec752a48bfb6da61a1212977c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:55:55.538177 7135 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1bcf2fec752a48bfb6da61a1212977c7 [term 1 LEADER]: Becoming Leader. State: Replica: 1bcf2fec752a48bfb6da61a1212977c7, State: Running, Role: LEADER
I20260808 16:55:55.538180 7131 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1bcf2fec752a48bfb6da61a1212977c7 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 16:55:55.538280 7135 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1bcf2fec752a48bfb6da61a1212977c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1bcf2fec752a48bfb6da61a1212977c7" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 42825 } }
I20260808 16:55:55.538642 7138 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1bcf2fec752a48bfb6da61a1212977c7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1bcf2fec752a48bfb6da61a1212977c7. Latest consensus state: current_term: 1 leader_uuid: "1bcf2fec752a48bfb6da61a1212977c7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1bcf2fec752a48bfb6da61a1212977c7" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 42825 } } }
I20260808 16:55:55.538729 7138 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1bcf2fec752a48bfb6da61a1212977c7 [sys.catalog]: This master's current role is: LEADER
I20260808 16:55:55.538812 7137 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1bcf2fec752a48bfb6da61a1212977c7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1bcf2fec752a48bfb6da61a1212977c7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1bcf2fec752a48bfb6da61a1212977c7" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 42825 } } }
I20260808 16:55:55.538940 7137 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1bcf2fec752a48bfb6da61a1212977c7 [sys.catalog]: This master's current role is: LEADER
I20260808 16:55:55.539196 7145 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 16:55:55.539395 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 7041
I20260808 16:55:55.539496 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786207919384338-8288-0/minicluster-data/master-0/wal/instance
I20260808 16:55:55.539651 8288 external_mini_cluster.cc:1428] Setting key 6da9ac7b6b2a5f5cd69d9cdb80ef1515
I20260808 16:55:55.539685 7145 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 16:55:55.541177 7145 catalog_manager.cc:1347] Generated new cluster ID: 31aacd7d9a8b4f3886c03d5b961d8db0
I20260808 16:55:55.541227 7145 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 16:55:55.545306 7166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:55.540193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46588 (local address 127.8.24.62:42825)
0808 16:55:55.540367 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0808 16:55:55.540374 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 16:55:55.540402 (+ 28us) server_negotiation.cc:407] Connection header received
0808 16:55:55.540505 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:55:55.540512 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:55:55.540575 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:55:55.540702 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:55:55.540908 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:55.542224 (+ 1316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:55.543322 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:55.543950 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:55.544070 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:55.544650 (+ 580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:55:55.544654 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:55:55.544655 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 16:55:55.544853 (+ 198us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:55:55.544896 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 16:55:55.544910 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:55:55.544977 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 16:55:55.545032 (+ 55us) server_negotiation.cc:299] Negotiation successful
0808 16:55:55.545108 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":78,"threads_started":1}
I20260808 16:55:55.553503 7145 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 16:55:55.554131 7145 catalog_manager.cc:1504] Loading token signing keys...
I20260808 16:55:55.562490 7145 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1bcf2fec752a48bfb6da61a1212977c7: Generated new TSK 0
I20260808 16:55:55.562637 7145 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 16:55:55.564457 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786207919384338-8288-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786207919384338-8288-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786207919384338-8288-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786207919384338-8288-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.1:0
--local_ip_for_outbound_sockets=127.8.24.1
--webserver_interface=127.8.24.1
--webserver_port=0
--tserver_master_addrs=127.8.24.62:42825
--builtin_ntp_servers=127.8.24.20:44391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 16:55:55.642958 7170 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:55:55.643131 7170 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:55:55.643151 7170 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:55:55.644227 7170 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:55:55.644265 7170 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:55:55.644289 7170 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.1
I20260808 16:55:55.645565 7170 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:44391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786207919384338-8288-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786207919384338-8288-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786207919384338-8288-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.24.1
--webserver_port=0
--tserver_master_addrs=127.8.24.62:42825
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7170
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.1
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786207919384338-8288-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:55:55.645787 7170 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:55:55.646046 7170 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:55:55.650903 7189 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:55:55.650899 7183 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:55:55.650921 7185 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:55:55.651453 7170 server_base.cc:1034] running on GCE node
I20260808 16:55:55.651659 7170 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:55:55.651871 7170 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:55:55.653020 7170 hybrid_clock.cc:648] HybridClock initialized: now 1786208155653006 us; error 30 us; skew 500 ppm
I20260808 16:55:55.654178 7170 webserver.cc:460] Webserver started at http://127.8.24.1:42309/ using document root <none> and password file <none>
I20260808 16:55:55.654443 7170 fs_manager.cc:359] Metadata directory not provided
I20260808 16:55:55.654495 7170 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:55:55.654634 7170 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:55:55.655593 7170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786207919384338-8288-0/minicluster-data/ts-0/data/instance:
uuid: "b8d733f5a5804f2a900f17643b750108"
format_stamp: "Formatted at 2026-08-08 16:55:55 on dist-test-slave-6zbq"
server_key: "68e835cba326c2bbf3c2629c4dbb2835"
server_key_iv: "e6ce0c6def72cf0ed2358c8660512113"
server_key_version: "encryptionkey@0"
I20260808 16:55:55.655921 7170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786207919384338-8288-0/minicluster-data/ts-0/wal/instance:
uuid: "b8d733f5a5804f2a900f17643b750108"
format_stamp: "Formatted at 2026-08-08 16:55:55 on dist-test-slave-6zbq"
server_key: "68e835cba326c2bbf3c2629c4dbb2835"
server_key_iv: "e6ce0c6def72cf0ed2358c8660512113"
server_key_version: "encryptionkey@0"
I20260808 16:55:55.657160 7170 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 16:55:55.657897 7197 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:55:55.658051 7170 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 16:55:55.658133 7170 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786207919384338-8288-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786207919384338-8288-0/minicluster-data/ts-0/wal
uuid: "b8d733f5a5804f2a900f17643b750108"
format_stamp: "Formatted at 2026-08-08 16:55:55 on dist-test-slave-6zbq"
server_key: "68e835cba326c2bbf3c2629c4dbb2835"
server_key_iv: "e6ce0c6def72cf0ed2358c8660512113"
server_key_version: "encryptionkey@0"
I20260808 16:55:55.658206 7170 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786207919384338-8288-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786207919384338-8288-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786207919384338-8288-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:55:55.671247 7170 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:55:55.671478 7170 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:55:55.671600 7170 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:55:55.671782 7170 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:55:55.672060 7170 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:55:55.672092 7170 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:55:55.672118 7170 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:55:55.672135 7170 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:55:55.677405 7170 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.1:46127
I20260808 16:55:55.677439 7366 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.1:46127 every 8 connection(s)
I20260808 16:55:55.677726 7170 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786207919384338-8288-0/minicluster-data/ts-0/data/info.pb
I20260808 16:55:55.679446 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 7170
I20260808 16:55:55.679555 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786207919384338-8288-0/minicluster-data/ts-0/wal/instance
I20260808 16:55:55.679764 8288 external_mini_cluster.cc:1428] Setting key 42c21fe1890ce891d9e848b66791021f
I20260808 16:55:55.680702 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786207919384338-8288-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786207919384338-8288-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786207919384338-8288-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786207919384338-8288-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.2:0
--local_ip_for_outbound_sockets=127.8.24.2
--webserver_interface=127.8.24.2
--webserver_port=0
--tserver_master_addrs=127.8.24.62:42825
--builtin_ntp_servers=127.8.24.20:44391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 16:55:55.682993 7166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:55.678329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:49863 (local address 127.8.24.62:42825)
0808 16:55:55.678395 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 16:55:55.678397 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:55:55.678601 (+ 204us) server_negotiation.cc:407] Connection header received
0808 16:55:55.679016 (+ 415us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:55:55.679017 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:55:55.679026 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:55:55.679078 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:55:55.679802 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:55.680408 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:55.681678 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:55.681869 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:55.681930 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:55.682750 (+ 820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:55:55.682751 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:55:55.682751 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 16:55:55.682757 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:55:55.682775 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 16:55:55.682780 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:55:55.682834 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 16:55:55.682890 (+ 56us) server_negotiation.cc:299] Negotiation successful
0808 16:55:55.682918 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 16:55:55.683125 7369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:55.678303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:42825 (local address 127.8.24.1:49863)
0808 16:55:55.678491 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:55.678526 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 16:55:55.678945 (+ 419us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:55:55.679126 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:55:55.679128 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:55:55.679136 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:55:55.679721 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:55.679728 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:55.680493 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:55.680494 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:55.681587 (+ 1093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:55.681590 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:55.681958 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:55.681960 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:55.682106 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:55.682617 (+ 511us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 16:55:55.682617 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:55:55.682642 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 16:55:55.682663 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:55:55.682845 (+ 182us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:55:55.682847 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:55:55.682849 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 16:55:55.682889 (+ 40us) client_negotiation.cc:240] Negotiation successful
0808 16:55:55.682959 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":90,"threads_started":1}
I20260808 16:55:55.683559 7367 heartbeater.cc:347] Connected to a master server at 127.8.24.62:42825
I20260808 16:55:55.683645 7367 heartbeater.cc:464] Registering TS with master...
I20260808 16:55:55.683810 7367 heartbeater.cc:511] Master 127.8.24.62:42825 requested a full tablet report, sending...
I20260808 16:55:55.684281 7085 ts_manager.cc:194] Registered new tserver with Master: b8d733f5a5804f2a900f17643b750108 (127.8.24.1:46127)
I20260808 16:55:55.684855 7085 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.24.1:49863
W20260808 16:55:55.760566 7370 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:55:55.760741 7370 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:55:55.760761 7370 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:55:55.761876 7370 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:55:55.761911 7370 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:55:55.761934 7370 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.2
I20260808 16:55:55.763263 7370 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:44391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786207919384338-8288-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786207919384338-8288-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786207919384338-8288-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.24.2
--webserver_port=0
--tserver_master_addrs=127.8.24.62:42825
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7370
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.2
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786207919384338-8288-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:55:55.763463 7370 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:55:55.763716 7370 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:55:55.768137 7381 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:55:55.768146 7380 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:55:55.768151 7383 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:55:55.768281 7370 server_base.cc:1034] running on GCE node
I20260808 16:55:55.768522 7370 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:55:55.768744 7370 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:55:55.769873 7370 hybrid_clock.cc:648] HybridClock initialized: now 1786208155769861 us; error 30 us; skew 500 ppm
I20260808 16:55:55.770929 7370 webserver.cc:460] Webserver started at http://127.8.24.2:43541/ using document root <none> and password file <none>
I20260808 16:55:55.771138 7370 fs_manager.cc:359] Metadata directory not provided
I20260808 16:55:55.771180 7370 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:55:55.771297 7370 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:55:55.772219 7370 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786207919384338-8288-0/minicluster-data/ts-1/data/instance:
uuid: "c8cd9b6d21e447ceb0edd8da369017b4"
format_stamp: "Formatted at 2026-08-08 16:55:55 on dist-test-slave-6zbq"
server_key: "88e6cf3391910d04cfcac8b1ebff0765"
server_key_iv: "ef61c06ab814c8730abc1577d9451b37"
server_key_version: "encryptionkey@0"
I20260808 16:55:55.772516 7370 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786207919384338-8288-0/minicluster-data/ts-1/wal/instance:
uuid: "c8cd9b6d21e447ceb0edd8da369017b4"
format_stamp: "Formatted at 2026-08-08 16:55:55 on dist-test-slave-6zbq"
server_key: "88e6cf3391910d04cfcac8b1ebff0765"
server_key_iv: "ef61c06ab814c8730abc1577d9451b37"
server_key_version: "encryptionkey@0"
I20260808 16:55:55.773653 7370 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 16:55:55.774266 7390 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:55:55.774427 7370 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 16:55:55.774507 7370 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786207919384338-8288-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786207919384338-8288-0/minicluster-data/ts-1/wal
uuid: "c8cd9b6d21e447ceb0edd8da369017b4"
format_stamp: "Formatted at 2026-08-08 16:55:55 on dist-test-slave-6zbq"
server_key: "88e6cf3391910d04cfcac8b1ebff0765"
server_key_iv: "ef61c06ab814c8730abc1577d9451b37"
server_key_version: "encryptionkey@0"
I20260808 16:55:55.774570 7370 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786207919384338-8288-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786207919384338-8288-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786207919384338-8288-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:55:55.794504 7370 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:55:55.794749 7370 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:55:55.794865 7370 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:55:55.795049 7370 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:55:55.795331 7370 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:55:55.795363 7370 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:55:55.795389 7370 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:55:55.795415 7370 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:55:55.800527 7370 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.2:40713
I20260808 16:55:55.800582 7558 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.2:40713 every 8 connection(s)
I20260808 16:55:55.800851 7370 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786207919384338-8288-0/minicluster-data/ts-1/data/info.pb
I20260808 16:55:55.805140 7166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:55.801464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:40481 (local address 127.8.24.62:42825)
0808 16:55:55.801538 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0808 16:55:55.801540 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:55:55.801705 (+ 165us) server_negotiation.cc:407] Connection header received
0808 16:55:55.802112 (+ 407us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:55:55.802113 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:55:55.802123 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:55:55.802170 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:55:55.802789 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:55.803258 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:55.804110 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:55.804288 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:55.804329 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:55.804974 (+ 645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:55:55.804975 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:55:55.804975 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 16:55:55.804978 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:55:55.804988 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 16:55:55.804991 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:55:55.805022 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0808 16:55:55.805068 (+ 46us) server_negotiation.cc:299] Negotiation successful
0808 16:55:55.805084 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 16:55:55.805330 7566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:55.801442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:42825 (local address 127.8.24.2:40481)
0808 16:55:55.801598 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:55.801627 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0808 16:55:55.802043 (+ 416us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:55:55.802218 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:55:55.802221 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:55:55.802228 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:55:55.802728 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:55.802732 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:55.803317 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:55.803318 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:55.804055 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:55.804057 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:55.804339 (+ 282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:55.804339 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:55.804432 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:55.804869 (+ 437us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 16:55:55.804869 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:55:55.804896 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 16:55:55.804917 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:55:55.805025 (+ 108us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:55:55.805026 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:55:55.805028 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 16:55:55.805068 (+ 40us) client_negotiation.cc:240] Negotiation successful
0808 16:55:55.805137 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":71,"threads_started":1}
I20260808 16:55:55.805751 7564 heartbeater.cc:347] Connected to a master server at 127.8.24.62:42825
I20260808 16:55:55.805828 7564 heartbeater.cc:464] Registering TS with master...
I20260808 16:55:55.805990 7564 heartbeater.cc:511] Master 127.8.24.62:42825 requested a full tablet report, sending...
I20260808 16:55:55.806318 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 7370
I20260808 16:55:55.806372 7085 ts_manager.cc:194] Registered new tserver with Master: c8cd9b6d21e447ceb0edd8da369017b4 (127.8.24.2:40713)
I20260808 16:55:55.806433 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786207919384338-8288-0/minicluster-data/ts-1/wal/instance
I20260808 16:55:55.806605 8288 external_mini_cluster.cc:1428] Setting key a2cce519bbbb272ee5e0e29bc1d52d4f
I20260808 16:55:55.806767 7085 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.24.2:40481
I20260808 16:55:55.807408 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786207919384338-8288-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786207919384338-8288-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786207919384338-8288-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786207919384338-8288-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.3:0
--local_ip_for_outbound_sockets=127.8.24.3
--webserver_interface=127.8.24.3
--webserver_port=0
--tserver_master_addrs=127.8.24.62:42825
--builtin_ntp_servers=127.8.24.20:44391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 16:55:55.885732 7569 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:55:55.885902 7569 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:55:55.885921 7569 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:55:55.886976 7569 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:55:55.887008 7569 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:55:55.887032 7569 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.3
I20260808 16:55:55.888273 7569 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:44391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786207919384338-8288-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786207919384338-8288-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786207919384338-8288-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.24.3
--webserver_port=0
--tserver_master_addrs=127.8.24.62:42825
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7569
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.3
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786207919384338-8288-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:55:55.888441 7569 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:55:55.888667 7569 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:55:55.893034 7579 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:55:55.893039 7578 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:55:55.893042 7582 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:55:55.893225 7569 server_base.cc:1034] running on GCE node
I20260808 16:55:55.893417 7569 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:55:55.893630 7569 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:55:55.894766 7569 hybrid_clock.cc:648] HybridClock initialized: now 1786208155894751 us; error 27 us; skew 500 ppm
I20260808 16:55:55.895852 7569 webserver.cc:460] Webserver started at http://127.8.24.3:41369/ using document root <none> and password file <none>
I20260808 16:55:55.896056 7569 fs_manager.cc:359] Metadata directory not provided
I20260808 16:55:55.896102 7569 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:55:55.896224 7569 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:55:55.897171 7569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786207919384338-8288-0/minicluster-data/ts-2/data/instance:
uuid: "68b570a20be54f3b8abcf0f9a8b5f02c"
format_stamp: "Formatted at 2026-08-08 16:55:55 on dist-test-slave-6zbq"
server_key: "0822c1e95c4308fc55d9c0f94ce2c5c8"
server_key_iv: "72ac26320a0ebf37754aa5e716232373"
server_key_version: "encryptionkey@0"
I20260808 16:55:55.897485 7569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786207919384338-8288-0/minicluster-data/ts-2/wal/instance:
uuid: "68b570a20be54f3b8abcf0f9a8b5f02c"
format_stamp: "Formatted at 2026-08-08 16:55:55 on dist-test-slave-6zbq"
server_key: "0822c1e95c4308fc55d9c0f94ce2c5c8"
server_key_iv: "72ac26320a0ebf37754aa5e716232373"
server_key_version: "encryptionkey@0"
I20260808 16:55:55.898649 7569 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 16:55:55.899279 7590 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:55:55.899423 7569 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 16:55:55.899500 7569 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786207919384338-8288-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786207919384338-8288-0/minicluster-data/ts-2/wal
uuid: "68b570a20be54f3b8abcf0f9a8b5f02c"
format_stamp: "Formatted at 2026-08-08 16:55:55 on dist-test-slave-6zbq"
server_key: "0822c1e95c4308fc55d9c0f94ce2c5c8"
server_key_iv: "72ac26320a0ebf37754aa5e716232373"
server_key_version: "encryptionkey@0"
I20260808 16:55:55.899559 7569 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786207919384338-8288-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786207919384338-8288-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786207919384338-8288-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:55:55.921303 7569 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:55:55.921551 7569 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:55:55.921666 7569 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:55:55.921864 7569 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:55:55.922144 7569 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:55:55.922183 7569 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:55:55.922209 7569 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:55:55.922226 7569 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:55:55.927161 7569 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.3:43657
I20260808 16:55:55.927197 7763 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.3:43657 every 8 connection(s)
I20260808 16:55:55.927479 7569 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786207919384338-8288-0/minicluster-data/ts-2/data/info.pb
I20260808 16:55:55.931739 7166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:55.928054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:35009 (local address 127.8.24.62:42825)
0808 16:55:55.928121 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0808 16:55:55.928123 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:55:55.928344 (+ 221us) server_negotiation.cc:407] Connection header received
0808 16:55:55.928731 (+ 387us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:55:55.928731 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:55:55.928741 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:55:55.928787 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:55:55.929345 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:55.929813 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:55.930746 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:55.930927 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:55.930970 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:55.931545 (+ 575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:55:55.931546 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:55:55.931546 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 16:55:55.931550 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:55:55.931564 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 16:55:55.931568 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:55:55.931603 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0808 16:55:55.931661 (+ 58us) server_negotiation.cc:299] Negotiation successful
0808 16:55:55.931683 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 16:55:55.931917 7766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:55.928040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:42825 (local address 127.8.24.3:35009)
0808 16:55:55.928215 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:55.928254 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 16:55:55.928659 (+ 405us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:55:55.928835 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:55:55.928838 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:55:55.928846 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:55:55.929274 (+ 428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:55.929278 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:55.929872 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:55.929873 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:55.930681 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:55.930684 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:55.930980 (+ 296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:55.930980 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:55.931074 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:55.931434 (+ 360us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 16:55:55.931434 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:55:55.931458 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 16:55:55.931478 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:55:55.931610 (+ 132us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:55:55.931612 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:55:55.931614 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 16:55:55.931656 (+ 42us) client_negotiation.cc:240] Negotiation successful
0808 16:55:55.931724 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":76,"threads_started":1}
I20260808 16:55:55.932044 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 7569
I20260808 16:55:55.932149 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786207919384338-8288-0/minicluster-data/ts-2/wal/instance
I20260808 16:55:55.932318 8288 external_mini_cluster.cc:1428] Setting key 2208ebc3766922d67ff3ead366c8efe2
I20260808 16:55:55.932366 7764 heartbeater.cc:347] Connected to a master server at 127.8.24.62:42825
I20260808 16:55:55.932452 7764 heartbeater.cc:464] Registering TS with master...
I20260808 16:55:55.932624 7764 heartbeater.cc:511] Master 127.8.24.62:42825 requested a full tablet report, sending...
I20260808 16:55:55.932960 7084 ts_manager.cc:194] Registered new tserver with Master: 68b570a20be54f3b8abcf0f9a8b5f02c (127.8.24.3:43657)
I20260808 16:55:55.933322 7084 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.24.3:35009
I20260808 16:55:55.943745 8288 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 16:55:55.947280 7166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:55.944989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52098 (local address 127.8.24.62:42825)
0808 16:55:55.945028 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 16:55:55.945030 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:55:55.945119 (+ 89us) server_negotiation.cc:407] Connection header received
0808 16:55:55.945161 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:55:55.945162 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:55:55.945171 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:55:55.945212 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:55:55.945338 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:55.945758 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:55.946461 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:55.946625 (+ 164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:55.946669 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:55.947100 (+ 431us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:55:55.947101 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:55:55.947101 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 16:55:55.947105 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:55:55.947117 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 16:55:55.947121 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:55:55.947168 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0808 16:55:55.947208 (+ 40us) server_negotiation.cc:299] Negotiation successful
0808 16:55:55.947225 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 16:55:55.948614 7084 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52098:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 16:55:55.950002 7084 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 16:55:55.955672 7778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:55.951855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.3:43657 (local address 127.0.0.1:39086)
0808 16:55:55.952154 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:55.952164 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 16:55:55.952209 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:55:55.952585 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:55:55.952586 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:55:55.952587 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:55:55.952620 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:55.952622 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:55.953730 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:55.953731 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:55.954794 (+ 1063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:55.954797 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:55.955453 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:55.955454 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:55.955504 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:55.955506 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 16:55:55.955545 (+ 39us) client_negotiation.cc:240] Negotiation successful
0808 16:55:55.955591 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"mutex_wait_us":3,"thread_start_us":53,"threads_started":1}
I20260808 16:55:55.955672 7773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:55.951855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.2:40713 (local address 127.0.0.1:59822)
0808 16:55:55.951998 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:55.952029 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0808 16:55:55.952087 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:55:55.952282 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:55:55.952285 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:55:55.952287 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:55:55.952343 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:55.952345 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:55.953392 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:55.953393 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:55.954521 (+ 1128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:55.954524 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:55.955416 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:55.955417 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:55.955489 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:55.955503 (+ 14us) client_negotiation.cc:769] Sending connection context
0808 16:55:55.955568 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 16:55:55.955597 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"mutex_wait_us":1,"thread_start_us":70,"threads_started":1}
I20260808 16:55:55.955706 7775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:55.951932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59822 (local address 127.8.24.2:40713)
0808 16:55:55.952086 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 16:55:55.952091 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 16:55:55.952101 (+ 10us) server_negotiation.cc:407] Connection header received
0808 16:55:55.952169 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:55:55.952171 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:55:55.952188 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:55:55.952262 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:55:55.952424 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:55.953327 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:55.954615 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:55.955325 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:55.955396 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:55.955476 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 16:55:55.955588 (+ 112us) server_negotiation.cc:299] Negotiation successful
0808 16:55:55.955627 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":79,"threads_started":1}
I20260808 16:55:55.956040 7779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:55.952168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39086 (local address 127.8.24.3:43657)
0808 16:55:55.952461 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0808 16:55:55.952465 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 16:55:55.952473 (+ 8us) server_negotiation.cc:407] Connection header received
0808 16:55:55.952502 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:55:55.952504 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:55:55.952516 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:55:55.952582 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:55:55.952672 (+ 90us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:55.953645 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:55.954869 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:55.955366 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:55.955854 (+ 488us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:55.955927 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 16:55:55.955946 (+ 19us) server_negotiation.cc:299] Negotiation successful
0808 16:55:55.955975 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":264,"thread_start_us":69,"threads_started":1}
I20260808 16:55:55.956485 7774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:55.952035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:46127 (local address 127.0.0.1:54852)
0808 16:55:55.952397 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:55.952406 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 16:55:55.952448 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:55:55.952911 (+ 463us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:55:55.952912 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:55:55.952914 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:55:55.952952 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:55.952953 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:55.954033 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:55.954034 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:55.954925 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:55.954928 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:55.956258 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:55.956259 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:55.956319 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:55.956323 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 16:55:55.956382 (+ 59us) client_negotiation.cc:240] Negotiation successful
0808 16:55:55.956419 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":73,"threads_started":1}
I20260808 16:55:55.956379 7458 tablet_service.cc:1467] Processing CreateTablet for tablet 5c7338612e144fd4b2274f9fed6323de (DEFAULT_TABLE table=test-table [id=356396431693463b8f3904e83057d9f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:55:55.956529 7782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:55.952390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54852 (local address 127.8.24.1:46127)
0808 16:55:55.952581 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0808 16:55:55.952586 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 16:55:55.952596 (+ 10us) server_negotiation.cc:407] Connection header received
0808 16:55:55.952757 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:55:55.952760 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:55:55.952775 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:55:55.952842 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:55:55.953025 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:55.953960 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:55.955005 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:55.955690 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:55.956134 (+ 444us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:55.956228 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 16:55:55.956411 (+ 183us) server_negotiation.cc:299] Negotiation successful
0808 16:55:55.956454 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":82,"threads_started":1}
I20260808 16:55:55.956624 7654 tablet_service.cc:1467] Processing CreateTablet for tablet 5c7338612e144fd4b2274f9fed6323de (DEFAULT_TABLE table=test-table [id=356396431693463b8f3904e83057d9f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:55:55.956725 7458 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c7338612e144fd4b2274f9fed6323de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:55:55.956924 7654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c7338612e144fd4b2274f9fed6323de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:55:55.957224 7267 tablet_service.cc:1467] Processing CreateTablet for tablet 5c7338612e144fd4b2274f9fed6323de (DEFAULT_TABLE table=test-table [id=356396431693463b8f3904e83057d9f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:55:55.957526 7267 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c7338612e144fd4b2274f9fed6323de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:55:55.959239 7783 tablet_bootstrap.cc:492] T 5c7338612e144fd4b2274f9fed6323de P c8cd9b6d21e447ceb0edd8da369017b4: Bootstrap starting.
I20260808 16:55:55.959895 7783 tablet_bootstrap.cc:654] T 5c7338612e144fd4b2274f9fed6323de P c8cd9b6d21e447ceb0edd8da369017b4: Neither blocks nor log segments found. Creating new log.
I20260808 16:55:55.960171 7783 log.cc:824] T 5c7338612e144fd4b2274f9fed6323de P c8cd9b6d21e447ceb0edd8da369017b4: Log is configured to *not* fsync() on all Append() calls
I20260808 16:55:55.960570 7788 tablet_bootstrap.cc:492] T 5c7338612e144fd4b2274f9fed6323de P b8d733f5a5804f2a900f17643b750108: Bootstrap starting.
I20260808 16:55:55.960681 7785 tablet_bootstrap.cc:492] T 5c7338612e144fd4b2274f9fed6323de P 68b570a20be54f3b8abcf0f9a8b5f02c: Bootstrap starting.
I20260808 16:55:55.960855 7783 tablet_bootstrap.cc:492] T 5c7338612e144fd4b2274f9fed6323de P c8cd9b6d21e447ceb0edd8da369017b4: No bootstrap required, opened a new log
I20260808 16:55:55.960922 7783 ts_tablet_manager.cc:1397] T 5c7338612e144fd4b2274f9fed6323de P c8cd9b6d21e447ceb0edd8da369017b4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 16:55:55.961201 7788 tablet_bootstrap.cc:654] T 5c7338612e144fd4b2274f9fed6323de P b8d733f5a5804f2a900f17643b750108: Neither blocks nor log segments found. Creating new log.
I20260808 16:55:55.961319 7785 tablet_bootstrap.cc:654] T 5c7338612e144fd4b2274f9fed6323de P 68b570a20be54f3b8abcf0f9a8b5f02c: Neither blocks nor log segments found. Creating new log.
I20260808 16:55:55.961476 7788 log.cc:824] T 5c7338612e144fd4b2274f9fed6323de P b8d733f5a5804f2a900f17643b750108: Log is configured to *not* fsync() on all Append() calls
I20260808 16:55:55.961627 7785 log.cc:824] T 5c7338612e144fd4b2274f9fed6323de P 68b570a20be54f3b8abcf0f9a8b5f02c: Log is configured to *not* fsync() on all Append() calls
I20260808 16:55:55.962174 7788 tablet_bootstrap.cc:492] T 5c7338612e144fd4b2274f9fed6323de P b8d733f5a5804f2a900f17643b750108: No bootstrap required, opened a new log
I20260808 16:55:55.962231 7788 ts_tablet_manager.cc:1397] T 5c7338612e144fd4b2274f9fed6323de P b8d733f5a5804f2a900f17643b750108: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 16:55:55.962430 7785 tablet_bootstrap.cc:492] T 5c7338612e144fd4b2274f9fed6323de P 68b570a20be54f3b8abcf0f9a8b5f02c: No bootstrap required, opened a new log
I20260808 16:55:55.962499 7785 ts_tablet_manager.cc:1397] T 5c7338612e144fd4b2274f9fed6323de P 68b570a20be54f3b8abcf0f9a8b5f02c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 16:55:55.963434 7783 raft_consensus.cc:348] T 5c7338612e144fd4b2274f9fed6323de P c8cd9b6d21e447ceb0edd8da369017b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68b570a20be54f3b8abcf0f9a8b5f02c" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 43657 } } peers { permanent_uuid: "c8cd9b6d21e447ceb0edd8da369017b4" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 40713 } } peers { permanent_uuid: "b8d733f5a5804f2a900f17643b750108" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 46127 } }
I20260808 16:55:55.963593 7783 raft_consensus.cc:374] T 5c7338612e144fd4b2274f9fed6323de P c8cd9b6d21e447ceb0edd8da369017b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:55:55.963624 7783 raft_consensus.cc:729] T 5c7338612e144fd4b2274f9fed6323de P c8cd9b6d21e447ceb0edd8da369017b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8cd9b6d21e447ceb0edd8da369017b4, State: Initialized, Role: FOLLOWER
I20260808 16:55:55.963755 7783 consensus_queue.cc:260] T 5c7338612e144fd4b2274f9fed6323de P c8cd9b6d21e447ceb0edd8da369017b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68b570a20be54f3b8abcf0f9a8b5f02c" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 43657 } } peers { permanent_uuid: "c8cd9b6d21e447ceb0edd8da369017b4" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 40713 } } peers { permanent_uuid: "b8d733f5a5804f2a900f17643b750108" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 46127 } }
I20260808 16:55:55.964016 7783 ts_tablet_manager.cc:1428] T 5c7338612e144fd4b2274f9fed6323de P c8cd9b6d21e447ceb0edd8da369017b4: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 16:55:55.964047 7564 heartbeater.cc:503] Master 127.8.24.62:42825 was elected leader, sending a full tablet report...
I20260808 16:55:55.964416 7788 raft_consensus.cc:348] T 5c7338612e144fd4b2274f9fed6323de P b8d733f5a5804f2a900f17643b750108 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68b570a20be54f3b8abcf0f9a8b5f02c" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 43657 } } peers { permanent_uuid: "c8cd9b6d21e447ceb0edd8da369017b4" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 40713 } } peers { permanent_uuid: "b8d733f5a5804f2a900f17643b750108" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 46127 } }
I20260808 16:55:55.964558 7788 raft_consensus.cc:374] T 5c7338612e144fd4b2274f9fed6323de P b8d733f5a5804f2a900f17643b750108 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:55:55.964591 7788 raft_consensus.cc:729] T 5c7338612e144fd4b2274f9fed6323de P b8d733f5a5804f2a900f17643b750108 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8d733f5a5804f2a900f17643b750108, State: Initialized, Role: FOLLOWER
I20260808 16:55:55.964704 7788 consensus_queue.cc:260] T 5c7338612e144fd4b2274f9fed6323de P b8d733f5a5804f2a900f17643b750108 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68b570a20be54f3b8abcf0f9a8b5f02c" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 43657 } } peers { permanent_uuid: "c8cd9b6d21e447ceb0edd8da369017b4" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 40713 } } peers { permanent_uuid: "b8d733f5a5804f2a900f17643b750108" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 46127 } }
I20260808 16:55:55.964718 7785 raft_consensus.cc:348] T 5c7338612e144fd4b2274f9fed6323de P 68b570a20be54f3b8abcf0f9a8b5f02c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68b570a20be54f3b8abcf0f9a8b5f02c" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 43657 } } peers { permanent_uuid: "c8cd9b6d21e447ceb0edd8da369017b4" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 40713 } } peers { permanent_uuid: "b8d733f5a5804f2a900f17643b750108" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 46127 } }
I20260808 16:55:55.964881 7785 raft_consensus.cc:374] T 5c7338612e144fd4b2274f9fed6323de P 68b570a20be54f3b8abcf0f9a8b5f02c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:55:55.964926 7785 raft_consensus.cc:729] T 5c7338612e144fd4b2274f9fed6323de P 68b570a20be54f3b8abcf0f9a8b5f02c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68b570a20be54f3b8abcf0f9a8b5f02c, State: Initialized, Role: FOLLOWER
I20260808 16:55:55.964939 7788 ts_tablet_manager.cc:1428] T 5c7338612e144fd4b2274f9fed6323de P b8d733f5a5804f2a900f17643b750108: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 16:55:55.965057 7367 heartbeater.cc:503] Master 127.8.24.62:42825 was elected leader, sending a full tablet report...
I20260808 16:55:55.965296 7785 consensus_queue.cc:260] T 5c7338612e144fd4b2274f9fed6323de P 68b570a20be54f3b8abcf0f9a8b5f02c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68b570a20be54f3b8abcf0f9a8b5f02c" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 43657 } } peers { permanent_uuid: "c8cd9b6d21e447ceb0edd8da369017b4" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 40713 } } peers { permanent_uuid: "b8d733f5a5804f2a900f17643b750108" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 46127 } }
I20260808 16:55:55.965533 7785 ts_tablet_manager.cc:1428] T 5c7338612e144fd4b2274f9fed6323de P 68b570a20be54f3b8abcf0f9a8b5f02c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 16:55:55.965562 7764 heartbeater.cc:503] Master 127.8.24.62:42825 was elected leader, sending a full tablet report...
I20260808 16:55:56.026137 7792 raft_consensus.cc:482] T 5c7338612e144fd4b2274f9fed6323de P c8cd9b6d21e447ceb0edd8da369017b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 16:55:56.026245 7792 raft_consensus.cc:504] T 5c7338612e144fd4b2274f9fed6323de P c8cd9b6d21e447ceb0edd8da369017b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68b570a20be54f3b8abcf0f9a8b5f02c" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 43657 } } peers { permanent_uuid: "c8cd9b6d21e447ceb0edd8da369017b4" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 40713 } } peers { permanent_uuid: "b8d733f5a5804f2a900f17643b750108" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 46127 } }
I20260808 16:55:56.026572 7792 leader_election.cc:288] T 5c7338612e144fd4b2274f9fed6323de P c8cd9b6d21e447ceb0edd8da369017b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 68b570a20be54f3b8abcf0f9a8b5f02c (127.8.24.3:43657), b8d733f5a5804f2a900f17643b750108 (127.8.24.1:46127)
I20260808 16:55:56.029023 7566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:56.026696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.3:43657 (local address 127.8.24.2:52205)
0808 16:55:56.026749 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:56.026761 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 16:55:56.026809 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:55:56.026941 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:55:56.026942 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:55:56.026944 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:55:56.027003 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:56.027004 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:56.027576 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:56.027577 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:56.028278 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:56.028291 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:56.028795 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:56.028796 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:56.028863 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:56.028866 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 16:55:56.028911 (+ 45us) client_negotiation.cc:240] Negotiation successful
0808 16:55:56.028951 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260808 16:55:56.029021 7779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:56.026754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:52205 (local address 127.8.24.3:43657)
0808 16:55:56.026822 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 16:55:56.026824 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:55:56.026833 (+ 9us) server_negotiation.cc:407] Connection header received
0808 16:55:56.026860 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:55:56.026861 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:55:56.026869 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:55:56.026920 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:55:56.027066 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:56.027505 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:56.028378 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:56.028721 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:56.028779 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:56.028796 (+ 17us) server_negotiation.cc:1035] Waiting for connection context
0808 16:55:56.028927 (+ 131us) server_negotiation.cc:299] Negotiation successful
0808 16:55:56.028961 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 16:55:56.029318 7782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:56.026869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:57637 (local address 127.8.24.1:46127)
0808 16:55:56.026915 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 16:55:56.026917 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:55:56.026993 (+ 76us) server_negotiation.cc:407] Connection header received
0808 16:55:56.027041 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:55:56.027042 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:55:56.027052 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:55:56.027093 (+ 41us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:55:56.027230 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:56.027677 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:56.028613 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:56.028970 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:56.029029 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:56.029038 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 16:55:56.029222 (+ 184us) server_negotiation.cc:299] Negotiation successful
0808 16:55:56.029262 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 16:55:56.029332 7796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:56.026786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:46127 (local address 127.8.24.2:57637)
0808 16:55:56.026937 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:56.026947 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 16:55:56.026994 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:55:56.027104 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:55:56.027105 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:55:56.027106 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:55:56.027171 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:56.027172 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:56.027752 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:56.027753 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:56.028534 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:55:56.028536 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:55:56.029047 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:55:56.029048 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:55:56.029120 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:56.029123 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 16:55:56.029209 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 16:55:56.029250 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":103,"threads_started":1}
I20260808 16:55:56.029325 7690 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c7338612e144fd4b2274f9fed6323de" candidate_uuid: "c8cd9b6d21e447ceb0edd8da369017b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68b570a20be54f3b8abcf0f9a8b5f02c" is_pre_election: true
I20260808 16:55:56.029450 7690 raft_consensus.cc:2454] T 5c7338612e144fd4b2274f9fed6323de P 68b570a20be54f3b8abcf0f9a8b5f02c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8cd9b6d21e447ceb0edd8da369017b4 in term 0.
I20260808 16:55:56.029663 7397 leader_election.cc:302] T 5c7338612e144fd4b2274f9fed6323de P c8cd9b6d21e447ceb0edd8da369017b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68b570a20be54f3b8abcf0f9a8b5f02c, c8cd9b6d21e447ceb0edd8da369017b4; no voters:
I20260808 16:55:56.029779 7792 raft_consensus.cc:2781] T 5c7338612e144fd4b2274f9fed6323de P c8cd9b6d21e447ceb0edd8da369017b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 16:55:56.029755 7303 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c7338612e144fd4b2274f9fed6323de" candidate_uuid: "c8cd9b6d21e447ceb0edd8da369017b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8d733f5a5804f2a900f17643b750108" is_pre_election: true
I20260808 16:55:56.029840 7792 raft_consensus.cc:482] T 5c7338612e144fd4b2274f9fed6323de P c8cd9b6d21e447ceb0edd8da369017b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 16:55:56.029868 7792 raft_consensus.cc:3037] T 5c7338612e144fd4b2274f9fed6323de P c8cd9b6d21e447ceb0edd8da369017b4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:55:56.029966 7303 raft_consensus.cc:2454] T 5c7338612e144fd4b2274f9fed6323de P b8d733f5a5804f2a900f17643b750108 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8cd9b6d21e447ceb0edd8da369017b4 in term 0.
I20260808 16:55:56.030747 7792 raft_consensus.cc:504] T 5c7338612e144fd4b2274f9fed6323de P c8cd9b6d21e447ceb0edd8da369017b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68b570a20be54f3b8abcf0f9a8b5f02c" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 43657 } } peers { permanent_uuid: "c8cd9b6d21e447ceb0edd8da369017b4" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 40713 } } peers { permanent_uuid: "b8d733f5a5804f2a900f17643b750108" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 46127 } }
I20260808 16:55:56.030864 7792 leader_election.cc:288] T 5c7338612e144fd4b2274f9fed6323de P c8cd9b6d21e447ceb0edd8da369017b4 [CANDIDATE]: Term 1 election: Requested vote from peers 68b570a20be54f3b8abcf0f9a8b5f02c (127.8.24.3:43657), b8d733f5a5804f2a900f17643b750108 (127.8.24.1:46127)
I20260808 16:55:56.031031 7690 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c7338612e144fd4b2274f9fed6323de" candidate_uuid: "c8cd9b6d21e447ceb0edd8da369017b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68b570a20be54f3b8abcf0f9a8b5f02c"
I20260808 16:55:56.031032 7303 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c7338612e144fd4b2274f9fed6323de" candidate_uuid: "c8cd9b6d21e447ceb0edd8da369017b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8d733f5a5804f2a900f17643b750108"
I20260808 16:55:56.031106 7690 raft_consensus.cc:3037] T 5c7338612e144fd4b2274f9fed6323de P 68b570a20be54f3b8abcf0f9a8b5f02c [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:55:56.031103 7303 raft_consensus.cc:3037] T 5c7338612e144fd4b2274f9fed6323de P b8d733f5a5804f2a900f17643b750108 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:55:56.031734 7690 raft_consensus.cc:2454] T 5c7338612e144fd4b2274f9fed6323de P 68b570a20be54f3b8abcf0f9a8b5f02c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8cd9b6d21e447ceb0edd8da369017b4 in term 1.
I20260808 16:55:56.031741 7303 raft_consensus.cc:2454] T 5c7338612e144fd4b2274f9fed6323de P b8d733f5a5804f2a900f17643b750108 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8cd9b6d21e447ceb0edd8da369017b4 in term 1.
I20260808 16:55:56.031874 7397 leader_election.cc:302] T 5c7338612e144fd4b2274f9fed6323de P c8cd9b6d21e447ceb0edd8da369017b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8d733f5a5804f2a900f17643b750108, c8cd9b6d21e447ceb0edd8da369017b4; no voters:
I20260808 16:55:56.031953 7792 raft_consensus.cc:2781] T 5c7338612e144fd4b2274f9fed6323de P c8cd9b6d21e447ceb0edd8da369017b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:55:56.032069 7792 raft_consensus.cc:686] T 5c7338612e144fd4b2274f9fed6323de P c8cd9b6d21e447ceb0edd8da369017b4 [term 1 LEADER]: Becoming Leader. State: Replica: c8cd9b6d21e447ceb0edd8da369017b4, State: Running, Role: LEADER
I20260808 16:55:56.032140 7792 consensus_queue.cc:237] T 5c7338612e144fd4b2274f9fed6323de P c8cd9b6d21e447ceb0edd8da369017b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68b570a20be54f3b8abcf0f9a8b5f02c" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 43657 } } peers { permanent_uuid: "c8cd9b6d21e447ceb0edd8da369017b4" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 40713 } } peers { permanent_uuid: "b8d733f5a5804f2a900f17643b750108" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 46127 } }
I20260808 16:55:56.032871 7085 catalog_manager.cc:5495] T 5c7338612e144fd4b2274f9fed6323de P c8cd9b6d21e447ceb0edd8da369017b4 reported cstate change: term changed from 0 to 1, leader changed from <none> to c8cd9b6d21e447ceb0edd8da369017b4 (127.8.24.2). New cstate: current_term: 1 leader_uuid: "c8cd9b6d21e447ceb0edd8da369017b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68b570a20be54f3b8abcf0f9a8b5f02c" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 43657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8cd9b6d21e447ceb0edd8da369017b4" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 40713 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b8d733f5a5804f2a900f17643b750108" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 46127 } health_report { overall_health: UNKNOWN } } }
I20260808 16:55:56.045106 7775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:56.043028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59826 (local address 127.8.24.2:40713)
0808 16:55:56.043079 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 16:55:56.043081 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:55:56.043089 (+ 8us) server_negotiation.cc:407] Connection header received
0808 16:55:56.043148 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:55:56.043149 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:55:56.043163 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:55:56.043216 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 16:55:56.043328 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:56.043738 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:56.044448 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:56.044618 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:56.044661 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:56.044795 (+ 134us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 16:55:56.044906 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 16:55:56.044974 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 16:55:56.045012 (+ 38us) server_negotiation.cc:299] Negotiation successful
0808 16:55:56.045028 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 16:55:56.046558 7303 raft_consensus.cc:1261] T 5c7338612e144fd4b2274f9fed6323de P b8d733f5a5804f2a900f17643b750108 [term 1 FOLLOWER]: Refusing update from remote peer c8cd9b6d21e447ceb0edd8da369017b4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:55:56.046558 7690 raft_consensus.cc:1261] T 5c7338612e144fd4b2274f9fed6323de P 68b570a20be54f3b8abcf0f9a8b5f02c [term 1 FOLLOWER]: Refusing update from remote peer c8cd9b6d21e447ceb0edd8da369017b4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:55:56.046867 7792 consensus_queue.cc:1035] T 5c7338612e144fd4b2274f9fed6323de P c8cd9b6d21e447ceb0edd8da369017b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68b570a20be54f3b8abcf0f9a8b5f02c" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 43657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:55:56.046985 7798 consensus_queue.cc:1035] T 5c7338612e144fd4b2274f9fed6323de P c8cd9b6d21e447ceb0edd8da369017b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8d733f5a5804f2a900f17643b750108" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 46127 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:55:56.049350 7805 mvcc.cc:205] Tried to move back new op lower bound from 7316308607164497920 to 7316308607108321280. Current Snapshot: MvccSnapshot[applied={T|T < 7316308607164497920}]
I20260808 16:55:56.049587 7804 mvcc.cc:205] Tried to move back new op lower bound from 7316308607164497920 to 7316308607108321280. Current Snapshot: MvccSnapshot[applied={T|T < 7316308607164497920}]
I20260808 16:55:56.051139 7085 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:52098:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 16:55:56.051230 7085 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:52098:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 16:55:56.051823 7085 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1bcf2fec752a48bfb6da61a1212977c7: Sending DeleteTablet for 3 replicas of tablet 5c7338612e144fd4b2274f9fed6323de
I20260808 16:55:56.052095 7458 tablet_service.cc:1514] Processing DeleteTablet for tablet 5c7338612e144fd4b2274f9fed6323de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:55:56 UTC) from {username='slave'} at 127.0.0.1:59822
I20260808 16:55:56.052122 7654 tablet_service.cc:1514] Processing DeleteTablet for tablet 5c7338612e144fd4b2274f9fed6323de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:55:56 UTC) from {username='slave'} at 127.0.0.1:39086
I20260808 16:55:56.052294 7817 tablet_replica.cc:330] T 5c7338612e144fd4b2274f9fed6323de P c8cd9b6d21e447ceb0edd8da369017b4: stopping tablet replica
I20260808 16:55:56.052393 7267 tablet_service.cc:1514] Processing DeleteTablet for tablet 5c7338612e144fd4b2274f9fed6323de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:55:56 UTC) from {username='slave'} at 127.0.0.1:54852
I20260808 16:55:56.052374 7817 raft_consensus.cc:2229] T 5c7338612e144fd4b2274f9fed6323de P c8cd9b6d21e447ceb0edd8da369017b4 [term 1 LEADER]: Raft consensus shutting down.
I20260808 16:55:56.052532 7817 raft_consensus.cc:2258] T 5c7338612e144fd4b2274f9fed6323de P c8cd9b6d21e447ceb0edd8da369017b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:55:56.052587 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 7170
I20260808 16:55:56.059028 7817 ts_tablet_manager.cc:1905] T 5c7338612e144fd4b2274f9fed6323de P c8cd9b6d21e447ceb0edd8da369017b4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:55:56.060518 7817 ts_tablet_manager.cc:1918] T 5c7338612e144fd4b2274f9fed6323de P c8cd9b6d21e447ceb0edd8da369017b4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 16:55:56.060587 7817 log.cc:1196] T 5c7338612e144fd4b2274f9fed6323de P c8cd9b6d21e447ceb0edd8da369017b4: Deleting WAL directory at /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786207919384338-8288-0/minicluster-data/ts-1/wal/wals/5c7338612e144fd4b2274f9fed6323de
I20260808 16:55:56.060849 7817 ts_tablet_manager.cc:1939] T 5c7338612e144fd4b2274f9fed6323de P c8cd9b6d21e447ceb0edd8da369017b4: Deleting consensus metadata
I20260808 16:55:56.061334 7063 catalog_manager.cc:4841] TS c8cd9b6d21e447ceb0edd8da369017b4 (127.8.24.2:40713): tablet 5c7338612e144fd4b2274f9fed6323de (table test-table [id=356396431693463b8f3904e83057d9f2]) successfully deleted
W20260808 16:55:56.065596 7064 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.24.1:46127 (error 108)
I20260808 16:55:56.065862 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 7370
I20260808 16:55:56.066018 7774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:56.065911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:46127 (local address 127.0.0.1:54864)
0808 16:55:56.065954 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0808 16:55:56.065991 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.1:46127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260808 16:55:56.066107 7064 catalog_manager.cc:4584] TS b8d733f5a5804f2a900f17643b750108 (127.8.24.1:46127): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5c7338612e144fd4b2274f9fed6323de: Network error: Client connection negotiation failed: client connection to 127.8.24.1:46127: connect: Connection refused (error 111)
I20260808 16:55:56.066222 7818 tablet_replica.cc:330] T 5c7338612e144fd4b2274f9fed6323de P 68b570a20be54f3b8abcf0f9a8b5f02c: stopping tablet replica
I20260808 16:55:56.066304 7818 raft_consensus.cc:2229] T 5c7338612e144fd4b2274f9fed6323de P 68b570a20be54f3b8abcf0f9a8b5f02c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 16:55:56.066437 7818 raft_consensus.cc:2258] T 5c7338612e144fd4b2274f9fed6323de P 68b570a20be54f3b8abcf0f9a8b5f02c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:55:56.075116 7818 ts_tablet_manager.cc:1905] T 5c7338612e144fd4b2274f9fed6323de P 68b570a20be54f3b8abcf0f9a8b5f02c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:55:56.076573 7818 ts_tablet_manager.cc:1918] T 5c7338612e144fd4b2274f9fed6323de P 68b570a20be54f3b8abcf0f9a8b5f02c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 16:55:56.076646 7818 log.cc:1196] T 5c7338612e144fd4b2274f9fed6323de P 68b570a20be54f3b8abcf0f9a8b5f02c: Deleting WAL directory at /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786207919384338-8288-0/minicluster-data/ts-2/wal/wals/5c7338612e144fd4b2274f9fed6323de
I20260808 16:55:56.076887 7818 ts_tablet_manager.cc:1939] T 5c7338612e144fd4b2274f9fed6323de P 68b570a20be54f3b8abcf0f9a8b5f02c: Deleting consensus metadata
I20260808 16:55:56.077277 7064 catalog_manager.cc:4841] TS 68b570a20be54f3b8abcf0f9a8b5f02c (127.8.24.3:43657): tablet 5c7338612e144fd4b2274f9fed6323de (table test-table [id=356396431693463b8f3904e83057d9f2]) successfully deleted
I20260808 16:55:56.078878 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 7569
I20260808 16:55:56.086855 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 7041
2026-08-08T16:55:56Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (691 ms)
[----------] 6 tests from EncryptionPolicyTest (8179 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786207919384338-8288-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 16:55:56 dist-test-slave-6zbq krb5kdc[7823](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 16:55:56 dist-test-slave-6zbq krb5kdc[7823](info): set up 2 sockets
Aug 08 16:55:56 dist-test-slave-6zbq krb5kdc[7823](info): commencing operation
krb5kdc: starting...
W20260808 16:55:57.205850 8288 mini_kdc.cc:121] Time spent starting KDC: real 1.096s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786207919384338-8288-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786207919384338-8288-0/krb5kdc/test-user.keytab.
Aug 08 16:55:57 dist-test-slave-6zbq krb5kdc[7823](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208157, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T16:55:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T16:55:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786207919384338-8288-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786207919384338-8288-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 16:55:57.320885 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786207919384338-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786207919384338-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786207919384338-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34733
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786207919384338-8288-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34733
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:55:57.399830 7839 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:55:57.400013 7839 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:55:57.400034 7839 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:55:57.401154 7839 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 16:55:57.401196 7839 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:55:57.401209 7839 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:55:57.401221 7839 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 16:55:57.401232 7839 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 16:55:57.402673 7839 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786207919384338-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786207919384338-8288-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34733
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786207919384338-8288-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34733
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7839
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786207919384338-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:55:57.402885 7839 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:55:57.403082 7839 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:55:57.407500 7849 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:55:57.407481 7850 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:55:57.407508 7853 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:55:57.407701 7839 server_base.cc:1034] running on GCE node
I20260808 16:55:57.407892 7839 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:55:57.408121 7839 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:55:57.409255 7839 hybrid_clock.cc:648] HybridClock initialized: now 1786208157409243 us; error 31 us; skew 500 ppm
Aug 08 16:55:57 dist-test-slave-6zbq krb5kdc[7823](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208157, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:55:57.411477 7839 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 16:55:57.411885 7839 webserver.cc:460] Webserver started at http://127.0.0.1:36399/ using document root <none> and password file <none>
I20260808 16:55:57.412066 7839 fs_manager.cc:359] Metadata directory not provided
I20260808 16:55:57.412108 7839 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:55:57.412222 7839 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:55:57.413152 7839 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786207919384338-8288-0/minicluster-data/master-0/data/instance:
uuid: "6fb4ea4c680046e1b14ada52e8d6ba59"
format_stamp: "Formatted at 2026-08-08 16:55:57 on dist-test-slave-6zbq"
server_key: "1852dacdc69bccd4f4307410dad81379"
server_key_iv: "ad5c4493ddda4b73a4d8bb221b4b3577"
server_key_version: "encryptionkey@0"
I20260808 16:55:57.413455 7839 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786207919384338-8288-0/minicluster-data/master-0/wal/instance:
uuid: "6fb4ea4c680046e1b14ada52e8d6ba59"
format_stamp: "Formatted at 2026-08-08 16:55:57 on dist-test-slave-6zbq"
server_key: "1852dacdc69bccd4f4307410dad81379"
server_key_iv: "ad5c4493ddda4b73a4d8bb221b4b3577"
server_key_version: "encryptionkey@0"
I20260808 16:55:57.414603 7839 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 16:55:57.415256 7866 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:55:57.415417 7839 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 16:55:57.415498 7839 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786207919384338-8288-0/minicluster-data/master-0/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786207919384338-8288-0/minicluster-data/master-0/wal
uuid: "6fb4ea4c680046e1b14ada52e8d6ba59"
format_stamp: "Formatted at 2026-08-08 16:55:57 on dist-test-slave-6zbq"
server_key: "1852dacdc69bccd4f4307410dad81379"
server_key_iv: "ad5c4493ddda4b73a4d8bb221b4b3577"
server_key_version: "encryptionkey@0"
I20260808 16:55:57.415550 7839 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786207919384338-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786207919384338-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786207919384338-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:55:57.429081 7839 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:55:57.430629 7839 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:55:57.430723 7839 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:55:57.434285 7839 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34733
I20260808 16:55:57.434316 7936 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34733 every 8 connection(s)
I20260808 16:55:57.434665 7839 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
I20260808 16:55:57.435016 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 7839
I20260808 16:55:57.435132 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786207919384338-8288-0/minicluster-data/master-0/wal/instance
I20260808 16:55:57.435299 8288 external_mini_cluster.cc:1428] Setting key 3278f0e7ecb1e6fede1a5e3af0f23953
I20260808 16:55:57.435642 7937 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:55:57.438664 7937 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6fb4ea4c680046e1b14ada52e8d6ba59: Bootstrap starting.
I20260808 16:55:57.439256 7937 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6fb4ea4c680046e1b14ada52e8d6ba59: Neither blocks nor log segments found. Creating new log.
I20260808 16:55:57.439538 7937 log.cc:824] T 00000000000000000000000000000000 P 6fb4ea4c680046e1b14ada52e8d6ba59: Log is configured to *not* fsync() on all Append() calls
I20260808 16:55:57.440230 7937 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6fb4ea4c680046e1b14ada52e8d6ba59: No bootstrap required, opened a new log
Aug 08 16:55:57 dist-test-slave-6zbq krb5kdc[7823](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208157, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 16:55:57.442102 7937 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6fb4ea4c680046e1b14ada52e8d6ba59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fb4ea4c680046e1b14ada52e8d6ba59" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34733 } }
I20260808 16:55:57.442234 7937 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6fb4ea4c680046e1b14ada52e8d6ba59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:55:57.442256 7937 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6fb4ea4c680046e1b14ada52e8d6ba59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fb4ea4c680046e1b14ada52e8d6ba59, State: Initialized, Role: FOLLOWER
I20260808 16:55:57.442402 7937 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6fb4ea4c680046e1b14ada52e8d6ba59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fb4ea4c680046e1b14ada52e8d6ba59" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34733 } }
I20260808 16:55:57.442468 7937 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6fb4ea4c680046e1b14ada52e8d6ba59 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 16:55:57.442489 7937 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6fb4ea4c680046e1b14ada52e8d6ba59 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 16:55:57.442533 7937 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6fb4ea4c680046e1b14ada52e8d6ba59 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:55:57.443009 7937 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6fb4ea4c680046e1b14ada52e8d6ba59 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fb4ea4c680046e1b14ada52e8d6ba59" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34733 } }
I20260808 16:55:57.443101 7937 leader_election.cc:302] T 00000000000000000000000000000000 P 6fb4ea4c680046e1b14ada52e8d6ba59 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6fb4ea4c680046e1b14ada52e8d6ba59; no voters:
I20260808 16:55:57.443264 7937 leader_election.cc:288] T 00000000000000000000000000000000 P 6fb4ea4c680046e1b14ada52e8d6ba59 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 16:55:57.443312 7945 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6fb4ea4c680046e1b14ada52e8d6ba59 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:55:57.443449 7945 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6fb4ea4c680046e1b14ada52e8d6ba59 [term 1 LEADER]: Becoming Leader. State: Replica: 6fb4ea4c680046e1b14ada52e8d6ba59, State: Running, Role: LEADER
I20260808 16:55:57.443524 7937 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6fb4ea4c680046e1b14ada52e8d6ba59 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 16:55:57.443538 7945 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6fb4ea4c680046e1b14ada52e8d6ba59 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fb4ea4c680046e1b14ada52e8d6ba59" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34733 } }
I20260808 16:55:57.443862 7946 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6fb4ea4c680046e1b14ada52e8d6ba59 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6fb4ea4c680046e1b14ada52e8d6ba59" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fb4ea4c680046e1b14ada52e8d6ba59" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34733 } } }
I20260808 16:55:57.443889 7947 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6fb4ea4c680046e1b14ada52e8d6ba59 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6fb4ea4c680046e1b14ada52e8d6ba59. Latest consensus state: current_term: 1 leader_uuid: "6fb4ea4c680046e1b14ada52e8d6ba59" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fb4ea4c680046e1b14ada52e8d6ba59" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34733 } } }
I20260808 16:55:57.444031 7947 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6fb4ea4c680046e1b14ada52e8d6ba59 [sys.catalog]: This master's current role is: LEADER
I20260808 16:55:57.444223 7946 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6fb4ea4c680046e1b14ada52e8d6ba59 [sys.catalog]: This master's current role is: LEADER
I20260808 16:55:57.444540 7958 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 16:55:57.445006 7958 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 16:55:57.446113 7940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:57.435757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60387 (local address 127.0.0.1:34733)
0808 16:55:57.435938 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0808 16:55:57.435946 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 16:55:57.435977 (+ 31us) server_negotiation.cc:407] Connection header received
0808 16:55:57.436637 (+ 660us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:55:57.436646 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:55:57.436723 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:55:57.436871 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:55:57.437431 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:57.438767 (+ 1336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:57.439487 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:57.440002 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:57.440108 (+ 106us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:57.442431 (+ 2323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:55:57.442435 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:55:57.442435 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:55:57.442444 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:55:57.444218 (+ 1774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:55:57.444679 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:55:57.444681 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:55:57.444681 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:55:57.444781 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:55:57.445066 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:55:57.445067 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:55:57.445067 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:55:57.445510 (+ 443us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:55:57.445676 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0808 16:55:57.445844 (+ 168us) server_negotiation.cc:299] Negotiation successful
0808 16:55:57.445928 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":82,"threads_started":1}
I20260808 16:55:57.447005 7958 catalog_manager.cc:1347] Generated new cluster ID: 0251746ce9584aa8a08c74d3d4c5722d
I20260808 16:55:57.447006 8288 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 16:55:57.447054 7958 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 16:55:57.456948 7940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:57.450427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38307 (local address 127.0.0.1:34733)
0808 16:55:57.450468 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 16:55:57.450470 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:55:57.450476 (+ 6us) server_negotiation.cc:407] Connection header received
0808 16:55:57.450494 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:55:57.450495 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:55:57.450503 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:55:57.450546 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:55:57.451132 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:57.451858 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:57.452787 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:57.453118 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:57.453185 (+ 67us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:57.454417 (+ 1232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:55:57.454419 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:55:57.454419 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:55:57.454426 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:55:57.455872 (+ 1446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:55:57.456238 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:55:57.456239 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:55:57.456239 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:55:57.456297 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:55:57.456477 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:55:57.456478 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:55:57.456478 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:55:57.456596 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:55:57.456688 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 16:55:57.456800 (+ 112us) server_negotiation.cc:299] Negotiation successful
0808 16:55:57.456861 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 16:55:57.458923 7958 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 16:55:57.459358 7958 catalog_manager.cc:1504] Loading token signing keys...
I20260808 16:55:57.463289 7958 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6fb4ea4c680046e1b14ada52e8d6ba59: Generated new TSK 0
I20260808 16:55:57.463446 7958 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 16:55:57.472448 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 7839
W20260808 16:55:57.488698 7821 connection.cc:535] client connection to 127.0.0.1:34733 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34733): Connection reset by peer (error 104)
2026-08-08T16:55:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1391 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786207919384338-8288-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 16:55:57 dist-test-slave-6zbq krb5kdc[7982](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 16:55:57 dist-test-slave-6zbq krb5kdc[7982](info): set up 2 sockets
Aug 08 16:55:57 dist-test-slave-6zbq krb5kdc[7982](info): commencing operation
krb5kdc: starting...
W20260808 16:55:58.601686 8288 mini_kdc.cc:121] Time spent starting KDC: real 1.100s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786207919384338-8288-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786207919384338-8288-0/krb5kdc/test-user.keytab.
Aug 08 16:55:58 dist-test-slave-6zbq krb5kdc[7982](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208158, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T16:55:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T16:55:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786207919384338-8288-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786207919384338-8288-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 16:55:58.709841 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786207919384338-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786207919384338-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786207919384338-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36389
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786207919384338-8288-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36389
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:55:58.786933 8001 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:55:58.787103 8001 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:55:58.787123 8001 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:55:58.788178 8001 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 16:55:58.788211 8001 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:55:58.788224 8001 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:55:58.788235 8001 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 16:55:58.788249 8001 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 16:55:58.789639 8001 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786207919384338-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786207919384338-8288-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36389
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786207919384338-8288-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36389
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8001
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786207919384338-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:55:58.789805 8001 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:55:58.789968 8001 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:55:58.794206 8015 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:55:58.794384 8001 server_base.cc:1034] running on GCE node
W20260808 16:55:58.794222 8012 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:55:58.794222 8013 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:55:58.794685 8001 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:55:58.794961 8001 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:55:58.796096 8001 hybrid_clock.cc:648] HybridClock initialized: now 1786208158796083 us; error 28 us; skew 500 ppm
Aug 08 16:55:58 dist-test-slave-6zbq krb5kdc[7982](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208158, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:55:58.798273 8001 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 16:55:58.798666 8001 webserver.cc:460] Webserver started at http://127.0.0.1:41023/ using document root <none> and password file <none>
I20260808 16:55:58.798878 8001 fs_manager.cc:359] Metadata directory not provided
I20260808 16:55:58.798925 8001 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:55:58.799044 8001 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:55:58.799986 8001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786207919384338-8288-0/minicluster-data/master-0/data/instance:
uuid: "1417818d16194b20ad373f23dbbaa883"
format_stamp: "Formatted at 2026-08-08 16:55:58 on dist-test-slave-6zbq"
server_key: "4482c03d1a87f1d769df9b037fbbd180"
server_key_iv: "f1e46cdc8c7ff3ecfac966ebb6804a28"
server_key_version: "encryptionkey@0"
I20260808 16:55:58.800302 8001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786207919384338-8288-0/minicluster-data/master-0/wal/instance:
uuid: "1417818d16194b20ad373f23dbbaa883"
format_stamp: "Formatted at 2026-08-08 16:55:58 on dist-test-slave-6zbq"
server_key: "4482c03d1a87f1d769df9b037fbbd180"
server_key_iv: "f1e46cdc8c7ff3ecfac966ebb6804a28"
server_key_version: "encryptionkey@0"
I20260808 16:55:58.801463 8001 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 16:55:58.802098 8028 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:55:58.802241 8001 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 16:55:58.802323 8001 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786207919384338-8288-0/minicluster-data/master-0/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786207919384338-8288-0/minicluster-data/master-0/wal
uuid: "1417818d16194b20ad373f23dbbaa883"
format_stamp: "Formatted at 2026-08-08 16:55:58 on dist-test-slave-6zbq"
server_key: "4482c03d1a87f1d769df9b037fbbd180"
server_key_iv: "f1e46cdc8c7ff3ecfac966ebb6804a28"
server_key_version: "encryptionkey@0"
I20260808 16:55:58.802413 8001 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786207919384338-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786207919384338-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786207919384338-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:55:58.815054 8001 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:55:58.816531 8001 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:55:58.816653 8001 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:55:58.820151 8001 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36389
I20260808 16:55:58.820189 8101 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36389 every 8 connection(s)
I20260808 16:55:58.820461 8001 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
I20260808 16:55:58.821305 8102 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:55:58.823343 8102 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1417818d16194b20ad373f23dbbaa883: Bootstrap starting.
I20260808 16:55:58.823637 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 8001
I20260808 16:55:58.823735 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786207919384338-8288-0/minicluster-data/master-0/wal/instance
I20260808 16:55:58.823889 8102 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1417818d16194b20ad373f23dbbaa883: Neither blocks nor log segments found. Creating new log.
I20260808 16:55:58.823899 8288 external_mini_cluster.cc:1428] Setting key 6ea8ea1730addbfd43f5b1295591fbaa
I20260808 16:55:58.824141 8102 log.cc:824] T 00000000000000000000000000000000 P 1417818d16194b20ad373f23dbbaa883: Log is configured to *not* fsync() on all Append() calls
I20260808 16:55:58.824951 8102 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1417818d16194b20ad373f23dbbaa883: No bootstrap required, opened a new log
I20260808 16:55:58.826694 8102 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1417818d16194b20ad373f23dbbaa883 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1417818d16194b20ad373f23dbbaa883" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36389 } }
I20260808 16:55:58.826820 8102 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1417818d16194b20ad373f23dbbaa883 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:55:58.826853 8102 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1417818d16194b20ad373f23dbbaa883 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1417818d16194b20ad373f23dbbaa883, State: Initialized, Role: FOLLOWER
I20260808 16:55:58.826974 8102 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1417818d16194b20ad373f23dbbaa883 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1417818d16194b20ad373f23dbbaa883" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36389 } }
I20260808 16:55:58.827056 8102 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1417818d16194b20ad373f23dbbaa883 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 16:55:58.827085 8102 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1417818d16194b20ad373f23dbbaa883 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 16:55:58.827124 8102 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1417818d16194b20ad373f23dbbaa883 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:55:58.827747 8102 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1417818d16194b20ad373f23dbbaa883 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1417818d16194b20ad373f23dbbaa883" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36389 } }
I20260808 16:55:58.827858 8102 leader_election.cc:302] T 00000000000000000000000000000000 P 1417818d16194b20ad373f23dbbaa883 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1417818d16194b20ad373f23dbbaa883; no voters:
I20260808 16:55:58.828022 8102 leader_election.cc:288] T 00000000000000000000000000000000 P 1417818d16194b20ad373f23dbbaa883 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 16:55:58.828055 8108 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1417818d16194b20ad373f23dbbaa883 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:55:58.828240 8108 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1417818d16194b20ad373f23dbbaa883 [term 1 LEADER]: Becoming Leader. State: Replica: 1417818d16194b20ad373f23dbbaa883, State: Running, Role: LEADER
I20260808 16:55:58.828344 8102 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1417818d16194b20ad373f23dbbaa883 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 16:55:58.828387 8108 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1417818d16194b20ad373f23dbbaa883 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1417818d16194b20ad373f23dbbaa883" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36389 } }
I20260808 16:55:58.828950 8105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:58.824315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35369 (local address 127.0.0.1:36389)
0808 16:55:58.824487 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0808 16:55:58.824495 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 16:55:58.824518 (+ 23us) server_negotiation.cc:407] Connection header received
0808 16:55:58.824900 (+ 382us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:55:58.824906 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:55:58.824947 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:55:58.825068 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:55:58.825305 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:58.826128 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:58.826936 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:58.827524 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:58.827637 (+ 113us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:58.828226 (+ 589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:55:58.828230 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:55:58.828230 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 16:55:58.828416 (+ 186us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:55:58.828461 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 16:55:58.828476 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:55:58.828598 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0808 16:55:58.828727 (+ 129us) server_negotiation.cc:299] Negotiation successful
0808 16:55:58.828751 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":72,"threads_started":1}
I20260808 16:55:58.829038 8111 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1417818d16194b20ad373f23dbbaa883 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1417818d16194b20ad373f23dbbaa883" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1417818d16194b20ad373f23dbbaa883" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36389 } } }
I20260808 16:55:58.829088 8111 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1417818d16194b20ad373f23dbbaa883 [sys.catalog]: This master's current role is: LEADER
I20260808 16:55:58.828941 8108 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1417818d16194b20ad373f23dbbaa883 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1417818d16194b20ad373f23dbbaa883. Latest consensus state: current_term: 1 leader_uuid: "1417818d16194b20ad373f23dbbaa883" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1417818d16194b20ad373f23dbbaa883" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36389 } } }
I20260808 16:55:58.829165 8108 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1417818d16194b20ad373f23dbbaa883 [sys.catalog]: This master's current role is: LEADER
I20260808 16:55:58.829388 8118 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 16:55:58.829941 8288 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 16:55:58.829990 8118 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 16:55:58.831754 8118 catalog_manager.cc:1347] Generated new cluster ID: 7cdc53a085dc4df6b86bb5f51ac8640f
I20260808 16:55:58.831809 8118 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 16:55:58.834705 8105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:55:58.831321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57395 (local address 127.0.0.1:36389)
0808 16:55:58.831359 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0808 16:55:58.831360 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 16:55:58.831484 (+ 124us) server_negotiation.cc:407] Connection header received
0808 16:55:58.831530 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:55:58.831531 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:55:58.831540 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:55:58.831595 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:55:58.831756 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:58.832450 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:58.833468 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:55:58.833760 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:55:58.833818 (+ 58us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:55:58.834422 (+ 604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:55:58.834423 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:55:58.834424 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 16:55:58.834427 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:55:58.834440 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 16:55:58.834444 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:55:58.834493 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 16:55:58.834550 (+ 57us) server_negotiation.cc:299] Negotiation successful
0808 16:55:58.834559 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 16:55:58.851652 8118 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 16:55:58.852113 8118 catalog_manager.cc:1504] Loading token signing keys...
I20260808 16:55:58.855382 8118 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1417818d16194b20ad373f23dbbaa883: Generated new TSK 0
I20260808 16:55:58.855513 8118 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 16:55:58.860708 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 8001
W20260808 16:55:58.887328 7979 connection.cc:535] client connection to 127.0.0.1:36389 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36389): Connection reset by peer (error 104)
2026-08-08T16:55:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1398 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786207919384338-8288-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 16:55:58 dist-test-slave-6zbq krb5kdc[8142](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 16:55:58 dist-test-slave-6zbq krb5kdc[8142](info): set up 2 sockets
Aug 08 16:55:58 dist-test-slave-6zbq krb5kdc[8142](info): commencing operation
krb5kdc: starting...
W20260808 16:56:00.007426 8288 mini_kdc.cc:121] Time spent starting KDC: real 1.107s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786207919384338-8288-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786207919384338-8288-0/krb5kdc/test-user.keytab.
Aug 08 16:56:00 dist-test-slave-6zbq krb5kdc[8142](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208160, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T16:56:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T16:56:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786207919384338-8288-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786207919384338-8288-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 16:56:00.114483 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786207919384338-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786207919384338-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786207919384338-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35295
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786207919384338-8288-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35295
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:56:00.194406 8162 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:56:00.194590 8162 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:56:00.194620 8162 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:56:00.195719 8162 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 16:56:00.195758 8162 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:56:00.195777 8162 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:56:00.195802 8162 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 16:56:00.195825 8162 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 16:56:00.197247 8162 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786207919384338-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786207919384338-8288-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35295
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786207919384338-8288-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35295
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8162
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786207919384338-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:56:00.197501 8162 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:56:00.197686 8162 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:56:00.202131 8170 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:56:00.202214 8162 server_base.cc:1034] running on GCE node
W20260808 16:56:00.202188 8169 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:00.202198 8173 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:56:00.202581 8162 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:56:00.202834 8162 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:56:00.203963 8162 hybrid_clock.cc:648] HybridClock initialized: now 1786208160203955 us; error 27 us; skew 500 ppm
Aug 08 16:56:00 dist-test-slave-6zbq krb5kdc[8142](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208160, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:56:00.210613 8162 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 16:56:00.210995 8162 webserver.cc:460] Webserver started at http://127.0.0.1:32927/ using document root <none> and password file <none>
I20260808 16:56:00.211208 8162 fs_manager.cc:359] Metadata directory not provided
I20260808 16:56:00.211251 8162 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:56:00.211378 8162 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:56:00.212329 8162 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786207919384338-8288-0/minicluster-data/master-0/data/instance:
uuid: "7fa0cb92d0b94bea9ccaa3a57bceb7bf"
format_stamp: "Formatted at 2026-08-08 16:56:00 on dist-test-slave-6zbq"
server_key: "61f226aa94925dff31e38f492cc0eb3e"
server_key_iv: "1c5ab434f9312d324aa0468b48be453f"
server_key_version: "encryptionkey@0"
I20260808 16:56:00.212637 8162 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786207919384338-8288-0/minicluster-data/master-0/wal/instance:
uuid: "7fa0cb92d0b94bea9ccaa3a57bceb7bf"
format_stamp: "Formatted at 2026-08-08 16:56:00 on dist-test-slave-6zbq"
server_key: "61f226aa94925dff31e38f492cc0eb3e"
server_key_iv: "1c5ab434f9312d324aa0468b48be453f"
server_key_version: "encryptionkey@0"
I20260808 16:56:00.213794 8162 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 16:56:00.214444 8186 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:00.214592 8162 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:00.214659 8162 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786207919384338-8288-0/minicluster-data/master-0/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786207919384338-8288-0/minicluster-data/master-0/wal
uuid: "7fa0cb92d0b94bea9ccaa3a57bceb7bf"
format_stamp: "Formatted at 2026-08-08 16:56:00 on dist-test-slave-6zbq"
server_key: "61f226aa94925dff31e38f492cc0eb3e"
server_key_iv: "1c5ab434f9312d324aa0468b48be453f"
server_key_version: "encryptionkey@0"
I20260808 16:56:00.214726 8162 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786207919384338-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786207919384338-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786207919384338-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:56:00.217756 8162 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:56:00.219707 8162 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:56:00.219826 8162 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:56:00.223479 8162 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35295
I20260808 16:56:00.223515 8251 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35295 every 8 connection(s)
I20260808 16:56:00.223773 8162 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
I20260808 16:56:00.224624 8252 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:56:00.226660 8252 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7fa0cb92d0b94bea9ccaa3a57bceb7bf: Bootstrap starting.
I20260808 16:56:00.227155 8252 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7fa0cb92d0b94bea9ccaa3a57bceb7bf: Neither blocks nor log segments found. Creating new log.
I20260808 16:56:00.227375 8252 log.cc:824] T 00000000000000000000000000000000 P 7fa0cb92d0b94bea9ccaa3a57bceb7bf: Log is configured to *not* fsync() on all Append() calls
I20260808 16:56:00.227952 8252 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7fa0cb92d0b94bea9ccaa3a57bceb7bf: No bootstrap required, opened a new log
I20260808 16:56:00.228389 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 8162
I20260808 16:56:00.228503 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786207919384338-8288-0/minicluster-data/master-0/wal/instance
I20260808 16:56:00.228708 8288 external_mini_cluster.cc:1428] Setting key 4bd80c80beb877d51bc9a56306eac114
I20260808 16:56:00.229871 8252 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7fa0cb92d0b94bea9ccaa3a57bceb7bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fa0cb92d0b94bea9ccaa3a57bceb7bf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35295 } }
I20260808 16:56:00.230017 8252 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7fa0cb92d0b94bea9ccaa3a57bceb7bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:56:00.230055 8252 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7fa0cb92d0b94bea9ccaa3a57bceb7bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fa0cb92d0b94bea9ccaa3a57bceb7bf, State: Initialized, Role: FOLLOWER
I20260808 16:56:00.230225 8252 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7fa0cb92d0b94bea9ccaa3a57bceb7bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fa0cb92d0b94bea9ccaa3a57bceb7bf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35295 } }
I20260808 16:56:00.230320 8252 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7fa0cb92d0b94bea9ccaa3a57bceb7bf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 16:56:00.230376 8252 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7fa0cb92d0b94bea9ccaa3a57bceb7bf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 16:56:00.230425 8252 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7fa0cb92d0b94bea9ccaa3a57bceb7bf [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:56:00.231014 8261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:00.229150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38075 (local address 127.0.0.1:35295)
0808 16:56:00.229314 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:00.229322 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:00.229345 (+ 23us) server_negotiation.cc:407] Connection header received
0808 16:56:00.229838 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:00.229843 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:00.229866 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:00.229965 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:00.230563 (+ 598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:00.230566 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:00.230567 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 16:56:00.230570 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:00.230604 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 16:56:00.230616 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:00.230669 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:00.230752 (+ 83us) server_negotiation.cc:299] Negotiation successful
0808 16:56:00.230784 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":69,"threads_started":1}
I20260808 16:56:00.231395 8252 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7fa0cb92d0b94bea9ccaa3a57bceb7bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fa0cb92d0b94bea9ccaa3a57bceb7bf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35295 } }
I20260808 16:56:00.231493 8252 leader_election.cc:302] T 00000000000000000000000000000000 P 7fa0cb92d0b94bea9ccaa3a57bceb7bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7fa0cb92d0b94bea9ccaa3a57bceb7bf; no voters:
I20260808 16:56:00.231652 8252 leader_election.cc:288] T 00000000000000000000000000000000 P 7fa0cb92d0b94bea9ccaa3a57bceb7bf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 16:56:00.231686 8263 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7fa0cb92d0b94bea9ccaa3a57bceb7bf [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:56:00.231793 8263 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7fa0cb92d0b94bea9ccaa3a57bceb7bf [term 1 LEADER]: Becoming Leader. State: Replica: 7fa0cb92d0b94bea9ccaa3a57bceb7bf, State: Running, Role: LEADER
I20260808 16:56:00.231820 8288 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 16:56:00.231889 8252 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7fa0cb92d0b94bea9ccaa3a57bceb7bf [sys.catalog]: configured and running, proceeding with master startup.
I20260808 16:56:00.231873 8263 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7fa0cb92d0b94bea9ccaa3a57bceb7bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fa0cb92d0b94bea9ccaa3a57bceb7bf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35295 } }
I20260808 16:56:00.232170 8263 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7fa0cb92d0b94bea9ccaa3a57bceb7bf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7fa0cb92d0b94bea9ccaa3a57bceb7bf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fa0cb92d0b94bea9ccaa3a57bceb7bf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35295 } } }
I20260808 16:56:00.232244 8263 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7fa0cb92d0b94bea9ccaa3a57bceb7bf [sys.catalog]: This master's current role is: LEADER
I20260808 16:56:00.232369 8263 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7fa0cb92d0b94bea9ccaa3a57bceb7bf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7fa0cb92d0b94bea9ccaa3a57bceb7bf. Latest consensus state: current_term: 1 leader_uuid: "7fa0cb92d0b94bea9ccaa3a57bceb7bf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fa0cb92d0b94bea9ccaa3a57bceb7bf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35295 } } }
I20260808 16:56:00.232425 8263 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7fa0cb92d0b94bea9ccaa3a57bceb7bf [sys.catalog]: This master's current role is: LEADER
I20260808 16:56:00.232534 8269 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 16:56:00.233228 8269 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 16:56:00.234539 8269 catalog_manager.cc:1347] Generated new cluster ID: a4e70bb0674c4c47be3ce7424fe777ab
I20260808 16:56:00.234630 8269 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 16:56:00.235311 8261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:00.233205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50267 (local address 127.0.0.1:35295)
0808 16:56:00.233240 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:00.233242 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:00.234488 (+ 1246us) server_negotiation.cc:407] Connection header received
0808 16:56:00.234569 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:00.234570 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:00.234575 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:00.234633 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:00.235117 (+ 484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:00.235118 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:00.235119 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 16:56:00.235123 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:00.235135 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 16:56:00.235139 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:00.235187 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:00.235244 (+ 57us) server_negotiation.cc:299] Negotiation successful
0808 16:56:00.235255 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 16:56:00.249655 8269 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 16:56:00.249792 8269 catalog_manager.cc:1504] Loading token signing keys...
I20260808 16:56:00.258308 8269 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7fa0cb92d0b94bea9ccaa3a57bceb7bf: Generated new TSK 0
I20260808 16:56:00.258478 8269 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 16:56:00.261839 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 8162
2026-08-08T16:56:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1386 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786207919384338-8288-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 16:56:00 dist-test-slave-6zbq krb5kdc[8315](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 16:56:00 dist-test-slave-6zbq krb5kdc[8315](info): set up 2 sockets
Aug 08 16:56:00 dist-test-slave-6zbq krb5kdc[8315](info): commencing operation
krb5kdc: starting...
W20260808 16:56:01.385936 8288 mini_kdc.cc:121] Time spent starting KDC: real 1.098s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786207919384338-8288-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786207919384338-8288-0/krb5kdc/test-user.keytab.
Aug 08 16:56:01 dist-test-slave-6zbq krb5kdc[8315](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208161, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T16:56:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T16:56:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786207919384338-8288-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786207919384338-8288-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 16:56:01.492715 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786207919384338-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786207919384338-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786207919384338-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42055
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786207919384338-8288-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:32805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42055
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:56:01.570498 8336 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:56:01.570672 8336 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:56:01.570703 8336 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:56:01.571779 8336 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 16:56:01.571820 8336 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:56:01.571838 8336 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:56:01.571854 8336 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 16:56:01.571870 8336 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 16:56:01.573295 8336 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:32805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786207919384338-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786207919384338-8288-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42055
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786207919384338-8288-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42055
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8336
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786207919384338-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:56:01.573518 8336 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:56:01.573712 8336 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 16:56:01.578178 8336 server_base.cc:1034] running on GCE node
W20260808 16:56:01.578203 8344 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:01.578150 8345 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:01.578135 8347 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:56:01.578589 8336 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:56:01.578837 8336 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:56:01.579967 8336 hybrid_clock.cc:648] HybridClock initialized: now 1786208161579960 us; error 27 us; skew 500 ppm
Aug 08 16:56:01 dist-test-slave-6zbq krb5kdc[8315](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208161, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:56:01.582149 8336 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 16:56:01.582569 8336 webserver.cc:460] Webserver started at http://127.0.0.1:38131/ using document root <none> and password file <none>
I20260808 16:56:01.582772 8336 fs_manager.cc:359] Metadata directory not provided
I20260808 16:56:01.582815 8336 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:56:01.582929 8336 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:56:01.583854 8336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786207919384338-8288-0/minicluster-data/master-0/data/instance:
uuid: "a0f3855b654f421a8e3476c9020e4dc3"
format_stamp: "Formatted at 2026-08-08 16:56:01 on dist-test-slave-6zbq"
server_key: "44f18d914c88528954d8ca8833f3e396"
server_key_iv: "6f31189dfca7c91b9b88a180c1e73a02"
server_key_version: "encryptionkey@0"
I20260808 16:56:01.584158 8336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786207919384338-8288-0/minicluster-data/master-0/wal/instance:
uuid: "a0f3855b654f421a8e3476c9020e4dc3"
format_stamp: "Formatted at 2026-08-08 16:56:01 on dist-test-slave-6zbq"
server_key: "44f18d914c88528954d8ca8833f3e396"
server_key_iv: "6f31189dfca7c91b9b88a180c1e73a02"
server_key_version: "encryptionkey@0"
I20260808 16:56:01.585323 8336 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 16:56:01.585975 8355 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:01.586169 8336 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:01.586254 8336 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786207919384338-8288-0/minicluster-data/master-0/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786207919384338-8288-0/minicluster-data/master-0/wal
uuid: "a0f3855b654f421a8e3476c9020e4dc3"
format_stamp: "Formatted at 2026-08-08 16:56:01 on dist-test-slave-6zbq"
server_key: "44f18d914c88528954d8ca8833f3e396"
server_key_iv: "6f31189dfca7c91b9b88a180c1e73a02"
server_key_version: "encryptionkey@0"
I20260808 16:56:01.586316 8336 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786207919384338-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786207919384338-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786207919384338-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:56:01.615165 8336 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:56:01.617117 8336 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:56:01.617221 8336 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:56:01.620735 8336 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42055
I20260808 16:56:01.620771 8437 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42055 every 8 connection(s)
I20260808 16:56:01.621061 8336 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
I20260808 16:56:01.622023 8438 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:56:01.624142 8438 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0f3855b654f421a8e3476c9020e4dc3: Bootstrap starting.
I20260808 16:56:01.624670 8438 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a0f3855b654f421a8e3476c9020e4dc3: Neither blocks nor log segments found. Creating new log.
I20260808 16:56:01.624925 8438 log.cc:824] T 00000000000000000000000000000000 P a0f3855b654f421a8e3476c9020e4dc3: Log is configured to *not* fsync() on all Append() calls
I20260808 16:56:01.625557 8438 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0f3855b654f421a8e3476c9020e4dc3: No bootstrap required, opened a new log
I20260808 16:56:01.626617 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 8336
I20260808 16:56:01.626721 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786207919384338-8288-0/minicluster-data/master-0/wal/instance
I20260808 16:56:01.626893 8288 external_mini_cluster.cc:1428] Setting key 6edba7bb66a278a37ef2e0a219d9c9bc
I20260808 16:56:01.627187 8438 raft_consensus.cc:348] T 00000000000000000000000000000000 P a0f3855b654f421a8e3476c9020e4dc3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0f3855b654f421a8e3476c9020e4dc3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42055 } }
I20260808 16:56:01.627374 8438 raft_consensus.cc:374] T 00000000000000000000000000000000 P a0f3855b654f421a8e3476c9020e4dc3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:56:01.627422 8438 raft_consensus.cc:729] T 00000000000000000000000000000000 P a0f3855b654f421a8e3476c9020e4dc3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0f3855b654f421a8e3476c9020e4dc3, State: Initialized, Role: FOLLOWER
I20260808 16:56:01.627537 8438 consensus_queue.cc:260] T 00000000000000000000000000000000 P a0f3855b654f421a8e3476c9020e4dc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0f3855b654f421a8e3476c9020e4dc3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42055 } }
I20260808 16:56:01.627600 8438 raft_consensus.cc:388] T 00000000000000000000000000000000 P a0f3855b654f421a8e3476c9020e4dc3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 16:56:01.627625 8438 raft_consensus.cc:482] T 00000000000000000000000000000000 P a0f3855b654f421a8e3476c9020e4dc3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 16:56:01.627655 8438 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a0f3855b654f421a8e3476c9020e4dc3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:56:01.628358 8438 raft_consensus.cc:504] T 00000000000000000000000000000000 P a0f3855b654f421a8e3476c9020e4dc3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0f3855b654f421a8e3476c9020e4dc3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42055 } }
I20260808 16:56:01.628444 8438 leader_election.cc:302] T 00000000000000000000000000000000 P a0f3855b654f421a8e3476c9020e4dc3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a0f3855b654f421a8e3476c9020e4dc3; no voters:
I20260808 16:56:01.628572 8438 leader_election.cc:288] T 00000000000000000000000000000000 P a0f3855b654f421a8e3476c9020e4dc3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 16:56:01.628607 8445 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a0f3855b654f421a8e3476c9020e4dc3 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:56:01.628736 8445 raft_consensus.cc:686] T 00000000000000000000000000000000 P a0f3855b654f421a8e3476c9020e4dc3 [term 1 LEADER]: Becoming Leader. State: Replica: a0f3855b654f421a8e3476c9020e4dc3, State: Running, Role: LEADER
I20260808 16:56:01.628788 8438 sys_catalog.cc:564] T 00000000000000000000000000000000 P a0f3855b654f421a8e3476c9020e4dc3 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 16:56:01.628829 8445 consensus_queue.cc:237] T 00000000000000000000000000000000 P a0f3855b654f421a8e3476c9020e4dc3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0f3855b654f421a8e3476c9020e4dc3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42055 } }
I20260808 16:56:01.629134 8445 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0f3855b654f421a8e3476c9020e4dc3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a0f3855b654f421a8e3476c9020e4dc3. Latest consensus state: current_term: 1 leader_uuid: "a0f3855b654f421a8e3476c9020e4dc3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0f3855b654f421a8e3476c9020e4dc3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42055 } } }
I20260808 16:56:01.629207 8445 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0f3855b654f421a8e3476c9020e4dc3 [sys.catalog]: This master's current role is: LEADER
I20260808 16:56:01.629349 8446 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0f3855b654f421a8e3476c9020e4dc3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a0f3855b654f421a8e3476c9020e4dc3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0f3855b654f421a8e3476c9020e4dc3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42055 } } }
I20260808 16:56:01.629397 8446 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0f3855b654f421a8e3476c9020e4dc3 [sys.catalog]: This master's current role is: LEADER
I20260808 16:56:01.629498 8453 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 16:56:01.630286 8453 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 16:56:01.631831 8453 catalog_manager.cc:1347] Generated new cluster ID: 31460ddbd9ec4800bc60ef5f220d1ec2
I20260808 16:56:01.631882 8453 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 16:56:01 dist-test-slave-6zbq krb5kdc[8315](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208161, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 16:56:01.639544 8444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:01.627327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40099 (local address 127.0.0.1:42055)
0808 16:56:01.628767 (+ 1440us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:01.628775 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:01.628818 (+ 43us) server_negotiation.cc:407] Connection header received
0808 16:56:01.628932 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:01.628939 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:01.628995 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:01.629130 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:01.629922 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:01.631200 (+ 1278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:01.632209 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:01.632830 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:01.632946 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:01.635525 (+ 2579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:01.635529 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:01.635530 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:56:01.635542 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:01.637980 (+ 2438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:01.638369 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:01.638372 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:01.638373 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:01.638492 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:01.638672 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:01.638673 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:01.638673 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:01.638969 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:01.639118 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:01.639189 (+ 71us) server_negotiation.cc:299] Negotiation successful
0808 16:56:01.639323 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1392,"thread_start_us":76,"threads_started":1}
I20260808 16:56:01.640543 8288 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 16:56:01.640638 8453 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 16:56:01.641265 8453 catalog_manager.cc:1504] Loading token signing keys...
I20260808 16:56:01.647413 8453 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a0f3855b654f421a8e3476c9020e4dc3: Generated new TSK 0
I20260808 16:56:01.647554 8453 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 16:56:01.648452 8444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:01.641956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48619 (local address 127.0.0.1:42055)
0808 16:56:01.641990 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:01.641992 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:01.642130 (+ 138us) server_negotiation.cc:407] Connection header received
0808 16:56:01.642159 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:01.642159 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:01.642170 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:01.642230 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:01.642800 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:01.643531 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:01.644477 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:01.644799 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:01.644862 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:01.645989 (+ 1127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:01.645991 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:01.645991 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:56:01.645998 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:01.647467 (+ 1469us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:01.647824 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:01.647825 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:01.647825 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:01.647883 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:01.648026 (+ 143us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:01.648027 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:01.648027 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:01.648136 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:01.648220 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:01.648294 (+ 74us) server_negotiation.cc:299] Negotiation successful
0808 16:56:01.648367 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 16:56:01.650090 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 8336
2026-08-08T16:56:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1385 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786207919384338-8288-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 16:56:01 dist-test-slave-6zbq krb5kdc[8482](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 16:56:01 dist-test-slave-6zbq krb5kdc[8482](info): set up 2 sockets
Aug 08 16:56:01 dist-test-slave-6zbq krb5kdc[8482](info): commencing operation
krb5kdc: starting...
W20260808 16:56:02.779549 8288 mini_kdc.cc:121] Time spent starting KDC: real 1.107s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786207919384338-8288-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786207919384338-8288-0/krb5kdc/test-user.keytab.
Aug 08 16:56:02 dist-test-slave-6zbq krb5kdc[8482](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208162, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T16:56:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T16:56:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786207919384338-8288-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786207919384338-8288-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 16:56:02.884135 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786207919384338-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786207919384338-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786207919384338-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37479
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786207919384338-8288-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44283
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37479
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:56:02.961337 8498 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:56:02.961505 8498 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:56:02.961526 8498 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:56:02.962622 8498 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 16:56:02.962659 8498 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:56:02.962680 8498 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:56:02.962704 8498 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 16:56:02.962728 8498 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 16:56:02.964149 8498 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44283
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786207919384338-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786207919384338-8288-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37479
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786207919384338-8288-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37479
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8498
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786207919384338-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:56:02.964355 8498 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:56:02.964546 8498 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:56:02.968883 8514 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:02.968923 8511 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:02.968950 8505 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:56:02.968948 8498 server_base.cc:1034] running on GCE node
I20260808 16:56:02.969298 8498 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:56:02.969533 8498 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:56:02.970672 8498 hybrid_clock.cc:648] HybridClock initialized: now 1786208162970659 us; error 28 us; skew 500 ppm
Aug 08 16:56:02 dist-test-slave-6zbq krb5kdc[8482](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208162, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:56:02.972872 8498 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 16:56:02.973251 8498 webserver.cc:460] Webserver started at http://127.0.0.1:41327/ using document root <none> and password file <none>
I20260808 16:56:02.973451 8498 fs_manager.cc:359] Metadata directory not provided
I20260808 16:56:02.973496 8498 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:56:02.973615 8498 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:56:02.974579 8498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786207919384338-8288-0/minicluster-data/master-0/data/instance:
uuid: "672b5041700045bc9114e63af9124350"
format_stamp: "Formatted at 2026-08-08 16:56:02 on dist-test-slave-6zbq"
server_key: "ce45f69c891085eb435c738f94e64c9c"
server_key_iv: "98fbb624eab40be31adb169d1aa6a325"
server_key_version: "encryptionkey@0"
I20260808 16:56:02.974884 8498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786207919384338-8288-0/minicluster-data/master-0/wal/instance:
uuid: "672b5041700045bc9114e63af9124350"
format_stamp: "Formatted at 2026-08-08 16:56:02 on dist-test-slave-6zbq"
server_key: "ce45f69c891085eb435c738f94e64c9c"
server_key_iv: "98fbb624eab40be31adb169d1aa6a325"
server_key_version: "encryptionkey@0"
I20260808 16:56:02.975999 8498 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 16:56:02.976603 8521 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:02.976748 8498 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 16:56:02.976826 8498 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786207919384338-8288-0/minicluster-data/master-0/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786207919384338-8288-0/minicluster-data/master-0/wal
uuid: "672b5041700045bc9114e63af9124350"
format_stamp: "Formatted at 2026-08-08 16:56:02 on dist-test-slave-6zbq"
server_key: "ce45f69c891085eb435c738f94e64c9c"
server_key_iv: "98fbb624eab40be31adb169d1aa6a325"
server_key_version: "encryptionkey@0"
I20260808 16:56:02.976883 8498 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786207919384338-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786207919384338-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786207919384338-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:56:02.984751 8498 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:56:02.986577 8498 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:56:02.986702 8498 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:56:02.990128 8498 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37479
I20260808 16:56:02.990159 8594 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37479 every 8 connection(s)
I20260808 16:56:02.990465 8498 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
I20260808 16:56:02.991324 8595 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:56:02.993252 8595 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 672b5041700045bc9114e63af9124350: Bootstrap starting.
I20260808 16:56:02.993759 8595 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 672b5041700045bc9114e63af9124350: Neither blocks nor log segments found. Creating new log.
I20260808 16:56:02.994004 8595 log.cc:824] T 00000000000000000000000000000000 P 672b5041700045bc9114e63af9124350: Log is configured to *not* fsync() on all Append() calls
I20260808 16:56:02.994608 8595 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 672b5041700045bc9114e63af9124350: No bootstrap required, opened a new log
I20260808 16:56:02.996053 8595 raft_consensus.cc:348] T 00000000000000000000000000000000 P 672b5041700045bc9114e63af9124350 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "672b5041700045bc9114e63af9124350" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37479 } }
I20260808 16:56:02.996176 8595 raft_consensus.cc:374] T 00000000000000000000000000000000 P 672b5041700045bc9114e63af9124350 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:56:02.996210 8595 raft_consensus.cc:729] T 00000000000000000000000000000000 P 672b5041700045bc9114e63af9124350 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 672b5041700045bc9114e63af9124350, State: Initialized, Role: FOLLOWER
I20260808 16:56:02.996348 8595 consensus_queue.cc:260] T 00000000000000000000000000000000 P 672b5041700045bc9114e63af9124350 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "672b5041700045bc9114e63af9124350" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37479 } }
I20260808 16:56:02.996409 8595 raft_consensus.cc:388] T 00000000000000000000000000000000 P 672b5041700045bc9114e63af9124350 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 16:56:02.996431 8595 raft_consensus.cc:482] T 00000000000000000000000000000000 P 672b5041700045bc9114e63af9124350 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 16:56:02.996475 8595 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 672b5041700045bc9114e63af9124350 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:56:02.996937 8595 raft_consensus.cc:504] T 00000000000000000000000000000000 P 672b5041700045bc9114e63af9124350 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "672b5041700045bc9114e63af9124350" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37479 } }
I20260808 16:56:02.997027 8595 leader_election.cc:302] T 00000000000000000000000000000000 P 672b5041700045bc9114e63af9124350 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 672b5041700045bc9114e63af9124350; no voters:
I20260808 16:56:02.997176 8595 leader_election.cc:288] T 00000000000000000000000000000000 P 672b5041700045bc9114e63af9124350 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 16:56:02.997211 8601 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 672b5041700045bc9114e63af9124350 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:56:02.997344 8601 raft_consensus.cc:686] T 00000000000000000000000000000000 P 672b5041700045bc9114e63af9124350 [term 1 LEADER]: Becoming Leader. State: Replica: 672b5041700045bc9114e63af9124350, State: Running, Role: LEADER
I20260808 16:56:02.997397 8595 sys_catalog.cc:564] T 00000000000000000000000000000000 P 672b5041700045bc9114e63af9124350 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 16:56:02.997428 8601 consensus_queue.cc:237] T 00000000000000000000000000000000 P 672b5041700045bc9114e63af9124350 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "672b5041700045bc9114e63af9124350" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37479 } }
I20260808 16:56:02.997892 8602 sys_catalog.cc:455] T 00000000000000000000000000000000 P 672b5041700045bc9114e63af9124350 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "672b5041700045bc9114e63af9124350" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "672b5041700045bc9114e63af9124350" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37479 } } }
I20260808 16:56:02.997924 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 8498
I20260808 16:56:02.997977 8602 sys_catalog.cc:458] T 00000000000000000000000000000000 P 672b5041700045bc9114e63af9124350 [sys.catalog]: This master's current role is: LEADER
I20260808 16:56:02.998036 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786207919384338-8288-0/minicluster-data/master-0/wal/instance
I20260808 16:56:02.998229 8288 external_mini_cluster.cc:1428] Setting key e46fdcb6a33aafc1697659a5becc66b6
I20260808 16:56:02.998869 8603 sys_catalog.cc:455] T 00000000000000000000000000000000 P 672b5041700045bc9114e63af9124350 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 672b5041700045bc9114e63af9124350. Latest consensus state: current_term: 1 leader_uuid: "672b5041700045bc9114e63af9124350" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "672b5041700045bc9114e63af9124350" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37479 } } }
I20260808 16:56:02.998965 8603 sys_catalog.cc:458] T 00000000000000000000000000000000 P 672b5041700045bc9114e63af9124350 [sys.catalog]: This master's current role is: LEADER
W20260808 16:56:02.999115 8628 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 672b5041700045bc9114e63af9124350: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 16:56:02.999161 8628 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 16:56:02.999239 8614 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 16:56:02.999408 8614 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 16:56:03.000592 8614 catalog_manager.cc:1347] Generated new cluster ID: 86f100c2798a40959004c916c18bd76f
I20260808 16:56:03.000638 8614 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 16:56:03.003937 8629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:02.998626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52769 (local address 127.0.0.1:37479)
0808 16:56:02.998869 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:02.998878 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:02.998915 (+ 37us) server_negotiation.cc:407] Connection header received
0808 16:56:02.999155 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:02.999162 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:02.999215 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:02.999572 (+ 357us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:02.999656 (+ 84us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:03.000976 (+ 1320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:03.002027 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:03.002646 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:03.002763 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:03.003306 (+ 543us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:03.003310 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:03.003310 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 16:56:03.003500 (+ 190us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:03.003536 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 16:56:03.003551 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:03.003624 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:03.003662 (+ 38us) server_negotiation.cc:299] Negotiation successful
0808 16:56:03.003723 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97,"thread_start_us":50,"threads_started":1}
I20260808 16:56:03.004810 8288 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 16:56:03.009318 8629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:03.006212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58589 (local address 127.0.0.1:37479)
0808 16:56:03.006243 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:03.006244 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:03.006319 (+ 75us) server_negotiation.cc:407] Connection header received
0808 16:56:03.006423 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:03.006423 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:03.006433 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:03.006496 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:03.006610 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:03.007313 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:03.008213 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:03.008495 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:03.008563 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:03.009095 (+ 532us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:03.009096 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:03.009096 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 16:56:03.009101 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:03.009114 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 16:56:03.009118 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:03.009167 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:03.009226 (+ 59us) server_negotiation.cc:299] Negotiation successful
0808 16:56:03.009249 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260808 16:56:03.010550 8614 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 16:56:03.011202 8614 catalog_manager.cc:1504] Loading token signing keys...
I20260808 16:56:03.015400 8614 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 672b5041700045bc9114e63af9124350: Generated new TSK 0
I20260808 16:56:03.015544 8614 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 16:56:03.019721 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 8498
2026-08-08T16:56:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1377 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786207919384338-8288-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 16:56:03 dist-test-slave-6zbq krb5kdc[8644](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 16:56:03 dist-test-slave-6zbq krb5kdc[8644](info): set up 2 sockets
Aug 08 16:56:03 dist-test-slave-6zbq krb5kdc[8644](info): commencing operation
krb5kdc: starting...
W20260808 16:56:04.168574 8288 mini_kdc.cc:121] Time spent starting KDC: real 1.119s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786207919384338-8288-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786207919384338-8288-0/krb5kdc/test-user.keytab.
Aug 08 16:56:04 dist-test-slave-6zbq krb5kdc[8644](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208164, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T16:56:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T16:56:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786207919384338-8288-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786207919384338-8288-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 16:56:04.274199 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786207919384338-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786207919384338-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786207919384338-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37019
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786207919384338-8288-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37019
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:56:04.351477 8664 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:56:04.351672 8664 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:56:04.351704 8664 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:56:04.352798 8664 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 16:56:04.352837 8664 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:56:04.352856 8664 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:56:04.352872 8664 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 16:56:04.352890 8664 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 16:56:04.354317 8664 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786207919384338-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786207919384338-8288-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37019
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786207919384338-8288-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37019
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8664
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786207919384338-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:56:04.354573 8664 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:56:04.354768 8664 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 16:56:04.359359 8664 server_base.cc:1034] running on GCE node
W20260808 16:56:04.359340 8671 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:04.359331 8669 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:04.359311 8673 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:56:04.359766 8664 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:56:04.360042 8664 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:56:04.361194 8664 hybrid_clock.cc:648] HybridClock initialized: now 1786208164361177 us; error 29 us; skew 500 ppm
Aug 08 16:56:04 dist-test-slave-6zbq krb5kdc[8644](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208164, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:56:04.363492 8664 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 16:56:04.363866 8664 webserver.cc:460] Webserver started at http://127.0.0.1:42753/ using document root <none> and password file <none>
I20260808 16:56:04.364055 8664 fs_manager.cc:359] Metadata directory not provided
I20260808 16:56:04.364100 8664 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:56:04.364221 8664 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:56:04.365141 8664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786207919384338-8288-0/minicluster-data/master-0/data/instance:
uuid: "f236257e8eeb42c4a2ce1400afa115b4"
format_stamp: "Formatted at 2026-08-08 16:56:04 on dist-test-slave-6zbq"
server_key: "7b26e1a4ab00589d9b7ab6d39760a805"
server_key_iv: "ca85cafccd98965a8095e049cad3a8ba"
server_key_version: "encryptionkey@0"
I20260808 16:56:04.365447 8664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786207919384338-8288-0/minicluster-data/master-0/wal/instance:
uuid: "f236257e8eeb42c4a2ce1400afa115b4"
format_stamp: "Formatted at 2026-08-08 16:56:04 on dist-test-slave-6zbq"
server_key: "7b26e1a4ab00589d9b7ab6d39760a805"
server_key_iv: "ca85cafccd98965a8095e049cad3a8ba"
server_key_version: "encryptionkey@0"
I20260808 16:56:04.366612 8664 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 16:56:04.367246 8680 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:04.367394 8664 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 16:56:04.367473 8664 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786207919384338-8288-0/minicluster-data/master-0/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786207919384338-8288-0/minicluster-data/master-0/wal
uuid: "f236257e8eeb42c4a2ce1400afa115b4"
format_stamp: "Formatted at 2026-08-08 16:56:04 on dist-test-slave-6zbq"
server_key: "7b26e1a4ab00589d9b7ab6d39760a805"
server_key_iv: "ca85cafccd98965a8095e049cad3a8ba"
server_key_version: "encryptionkey@0"
I20260808 16:56:04.367529 8664 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786207919384338-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786207919384338-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786207919384338-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:56:04.370519 8664 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:56:04.372045 8664 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:56:04.372165 8664 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:56:04.375753 8664 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37019
I20260808 16:56:04.375788 8763 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37019 every 8 connection(s)
I20260808 16:56:04.376051 8664 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
I20260808 16:56:04.376906 8764 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:56:04.377876 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 8664
I20260808 16:56:04.378003 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786207919384338-8288-0/minicluster-data/master-0/wal/instance
I20260808 16:56:04.378197 8288 external_mini_cluster.cc:1428] Setting key 510ccb8e812a72b7b1509cf9bd4a822f
I20260808 16:56:04.379248 8764 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f236257e8eeb42c4a2ce1400afa115b4: Bootstrap starting.
I20260808 16:56:04.379796 8764 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f236257e8eeb42c4a2ce1400afa115b4: Neither blocks nor log segments found. Creating new log.
I20260808 16:56:04.380096 8764 log.cc:824] T 00000000000000000000000000000000 P f236257e8eeb42c4a2ce1400afa115b4: Log is configured to *not* fsync() on all Append() calls
I20260808 16:56:04.380563 8770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:04.378681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38189 (local address 127.0.0.1:37019)
0808 16:56:04.378846 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:04.378855 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:04.378878 (+ 23us) server_negotiation.cc:407] Connection header received
0808 16:56:04.379244 (+ 366us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:04.379250 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:04.379274 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:04.379391 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:04.380155 (+ 764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:04.380159 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:04.380159 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 16:56:04.380164 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:04.380206 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 16:56:04.380220 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:04.380317 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:04.380350 (+ 33us) server_negotiation.cc:299] Negotiation successful
0808 16:56:04.380387 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":70,"threads_started":1}
I20260808 16:56:04.380788 8764 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f236257e8eeb42c4a2ce1400afa115b4: No bootstrap required, opened a new log
I20260808 16:56:04.381444 8288 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 16:56:04.383066 8764 raft_consensus.cc:348] T 00000000000000000000000000000000 P f236257e8eeb42c4a2ce1400afa115b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f236257e8eeb42c4a2ce1400afa115b4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37019 } }
I20260808 16:56:04.383229 8764 raft_consensus.cc:374] T 00000000000000000000000000000000 P f236257e8eeb42c4a2ce1400afa115b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:56:04.383276 8764 raft_consensus.cc:729] T 00000000000000000000000000000000 P f236257e8eeb42c4a2ce1400afa115b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f236257e8eeb42c4a2ce1400afa115b4, State: Initialized, Role: FOLLOWER
I20260808 16:56:04.383399 8764 consensus_queue.cc:260] T 00000000000000000000000000000000 P f236257e8eeb42c4a2ce1400afa115b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f236257e8eeb42c4a2ce1400afa115b4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37019 } }
I20260808 16:56:04.383502 8764 raft_consensus.cc:388] T 00000000000000000000000000000000 P f236257e8eeb42c4a2ce1400afa115b4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 16:56:04.383548 8764 raft_consensus.cc:482] T 00000000000000000000000000000000 P f236257e8eeb42c4a2ce1400afa115b4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 16:56:04.383607 8764 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f236257e8eeb42c4a2ce1400afa115b4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:56:04.383903 8770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:04.382952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45269 (local address 127.0.0.1:37019)
0808 16:56:04.382988 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:04.382990 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:04.383084 (+ 94us) server_negotiation.cc:407] Connection header received
0808 16:56:04.383162 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:04.383163 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:04.383167 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:04.383206 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:04.383728 (+ 522us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:04.383729 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:04.383729 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 16:56:04.383734 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:04.383745 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 16:56:04.383749 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:04.383791 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:04.383849 (+ 58us) server_negotiation.cc:299] Negotiation successful
0808 16:56:04.383858 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 16:56:04.384207 8764 raft_consensus.cc:504] T 00000000000000000000000000000000 P f236257e8eeb42c4a2ce1400afa115b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f236257e8eeb42c4a2ce1400afa115b4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37019 } }
I20260808 16:56:04.384311 8764 leader_election.cc:302] T 00000000000000000000000000000000 P f236257e8eeb42c4a2ce1400afa115b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f236257e8eeb42c4a2ce1400afa115b4; no voters:
I20260808 16:56:04.384452 8764 leader_election.cc:288] T 00000000000000000000000000000000 P f236257e8eeb42c4a2ce1400afa115b4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 16:56:04.384482 8783 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f236257e8eeb42c4a2ce1400afa115b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:56:04.384606 8783 raft_consensus.cc:686] T 00000000000000000000000000000000 P f236257e8eeb42c4a2ce1400afa115b4 [term 1 LEADER]: Becoming Leader. State: Replica: f236257e8eeb42c4a2ce1400afa115b4, State: Running, Role: LEADER
I20260808 16:56:04.384677 8764 sys_catalog.cc:564] T 00000000000000000000000000000000 P f236257e8eeb42c4a2ce1400afa115b4 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 16:56:04.384720 8783 consensus_queue.cc:237] T 00000000000000000000000000000000 P f236257e8eeb42c4a2ce1400afa115b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f236257e8eeb42c4a2ce1400afa115b4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37019 } }
I20260808 16:56:04.385043 8786 sys_catalog.cc:455] T 00000000000000000000000000000000 P f236257e8eeb42c4a2ce1400afa115b4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f236257e8eeb42c4a2ce1400afa115b4. Latest consensus state: current_term: 1 leader_uuid: "f236257e8eeb42c4a2ce1400afa115b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f236257e8eeb42c4a2ce1400afa115b4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37019 } } }
I20260808 16:56:04.385154 8786 sys_catalog.cc:458] T 00000000000000000000000000000000 P f236257e8eeb42c4a2ce1400afa115b4 [sys.catalog]: This master's current role is: LEADER
I20260808 16:56:04.385038 8784 sys_catalog.cc:455] T 00000000000000000000000000000000 P f236257e8eeb42c4a2ce1400afa115b4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f236257e8eeb42c4a2ce1400afa115b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f236257e8eeb42c4a2ce1400afa115b4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37019 } } }
I20260808 16:56:04.385355 8784 sys_catalog.cc:458] T 00000000000000000000000000000000 P f236257e8eeb42c4a2ce1400afa115b4 [sys.catalog]: This master's current role is: LEADER
I20260808 16:56:04.385622 8792 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 16:56:04.386044 8792 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 16:56:04.387374 8792 catalog_manager.cc:1347] Generated new cluster ID: f4acceb8d9604ed3ad5a9c9d0f0950f3
I20260808 16:56:04.387425 8792 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 16:56:04.403522 8792 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 16:56:04.403656 8792 catalog_manager.cc:1504] Loading token signing keys...
I20260808 16:56:04.410115 8792 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f236257e8eeb42c4a2ce1400afa115b4: Generated new TSK 0
I20260808 16:56:04.410254 8792 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 16:56:04.418155 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 8664
2026-08-08T16:56:04Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1392 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786207919384338-8288-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 16:56:04 dist-test-slave-6zbq krb5kdc[8811](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 16:56:04 dist-test-slave-6zbq krb5kdc[8811](info): set up 2 sockets
Aug 08 16:56:04 dist-test-slave-6zbq krb5kdc[8811](info): commencing operation
krb5kdc: starting...
W20260808 16:56:05.547045 8288 mini_kdc.cc:121] Time spent starting KDC: real 1.104s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786207919384338-8288-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786207919384338-8288-0/krb5kdc/test-user.keytab.
Aug 08 16:56:05 dist-test-slave-6zbq krb5kdc[8811](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208165, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T16:56:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T16:56:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786207919384338-8288-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786207919384338-8288-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 16:56:05.664767 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786207919384338-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786207919384338-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786207919384338-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35831
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786207919384338-8288-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35831
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:56:05.741822 8831 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:56:05.742010 8831 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:56:05.742039 8831 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:56:05.743145 8831 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 16:56:05.743192 8831 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:56:05.743211 8831 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:56:05.743227 8831 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 16:56:05.743243 8831 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 16:56:05.744731 8831 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786207919384338-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786207919384338-8288-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35831
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786207919384338-8288-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35831
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8831
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786207919384338-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:56:05.744968 8831 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:56:05.745165 8831 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:56:05.749477 8837 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:05.749490 8838 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:56:05.749539 8831 server_base.cc:1034] running on GCE node
W20260808 16:56:05.749502 8844 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:56:05.749855 8831 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:56:05.750087 8831 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:56:05.751246 8831 hybrid_clock.cc:648] HybridClock initialized: now 1786208165751228 us; error 33 us; skew 500 ppm
Aug 08 16:56:05 dist-test-slave-6zbq krb5kdc[8811](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208165, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:56:05.753530 8831 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 16:56:05.753901 8831 webserver.cc:460] Webserver started at http://127.0.0.1:44701/ using document root <none> and password file <none>
I20260808 16:56:05.754081 8831 fs_manager.cc:359] Metadata directory not provided
I20260808 16:56:05.754120 8831 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:56:05.754208 8831 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:56:05.755162 8831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786207919384338-8288-0/minicluster-data/master-0/data/instance:
uuid: "7234a0e1e66141dbb6e4535d5d68bb33"
format_stamp: "Formatted at 2026-08-08 16:56:05 on dist-test-slave-6zbq"
server_key: "7cb28608e40b415ddbbaa63a67350afc"
server_key_iv: "952f402bfaa1b43324b8d0e2ceec9eb7"
server_key_version: "encryptionkey@0"
I20260808 16:56:05.755451 8831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786207919384338-8288-0/minicluster-data/master-0/wal/instance:
uuid: "7234a0e1e66141dbb6e4535d5d68bb33"
format_stamp: "Formatted at 2026-08-08 16:56:05 on dist-test-slave-6zbq"
server_key: "7cb28608e40b415ddbbaa63a67350afc"
server_key_iv: "952f402bfaa1b43324b8d0e2ceec9eb7"
server_key_version: "encryptionkey@0"
I20260808 16:56:05.756563 8831 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 16:56:05.757158 8852 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:05.757305 8831 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 16:56:05.757385 8831 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786207919384338-8288-0/minicluster-data/master-0/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786207919384338-8288-0/minicluster-data/master-0/wal
uuid: "7234a0e1e66141dbb6e4535d5d68bb33"
format_stamp: "Formatted at 2026-08-08 16:56:05 on dist-test-slave-6zbq"
server_key: "7cb28608e40b415ddbbaa63a67350afc"
server_key_iv: "952f402bfaa1b43324b8d0e2ceec9eb7"
server_key_version: "encryptionkey@0"
I20260808 16:56:05.757442 8831 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786207919384338-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786207919384338-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786207919384338-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:56:05.776717 8831 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:56:05.778286 8831 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:56:05.778441 8831 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:56:05.783754 8831 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35831
I20260808 16:56:05.783794 8928 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35831 every 8 connection(s)
I20260808 16:56:05.784106 8831 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
I20260808 16:56:05.785017 8930 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:56:05.787250 8930 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7234a0e1e66141dbb6e4535d5d68bb33: Bootstrap starting.
I20260808 16:56:05.787781 8930 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7234a0e1e66141dbb6e4535d5d68bb33: Neither blocks nor log segments found. Creating new log.
I20260808 16:56:05.788004 8930 log.cc:824] T 00000000000000000000000000000000 P 7234a0e1e66141dbb6e4535d5d68bb33: Log is configured to *not* fsync() on all Append() calls
I20260808 16:56:05.788477 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 8831
I20260808 16:56:05.788602 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786207919384338-8288-0/minicluster-data/master-0/wal/instance
I20260808 16:56:05.788741 8930 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7234a0e1e66141dbb6e4535d5d68bb33: No bootstrap required, opened a new log
I20260808 16:56:05.788793 8288 external_mini_cluster.cc:1428] Setting key 5698ac22ce216b77f1908c104d1f20d6
I20260808 16:56:05.790908 8930 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7234a0e1e66141dbb6e4535d5d68bb33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7234a0e1e66141dbb6e4535d5d68bb33" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35831 } }
I20260808 16:56:05.791052 8930 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7234a0e1e66141dbb6e4535d5d68bb33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:56:05.791074 8930 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7234a0e1e66141dbb6e4535d5d68bb33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7234a0e1e66141dbb6e4535d5d68bb33, State: Initialized, Role: FOLLOWER
I20260808 16:56:05.791182 8930 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7234a0e1e66141dbb6e4535d5d68bb33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7234a0e1e66141dbb6e4535d5d68bb33" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35831 } }
I20260808 16:56:05.791249 8930 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7234a0e1e66141dbb6e4535d5d68bb33 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 16:56:05.791277 8930 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7234a0e1e66141dbb6e4535d5d68bb33 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 16:56:05.791323 8930 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7234a0e1e66141dbb6e4535d5d68bb33 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:56:05.791824 8930 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7234a0e1e66141dbb6e4535d5d68bb33 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7234a0e1e66141dbb6e4535d5d68bb33" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35831 } }
I20260808 16:56:05.791913 8930 leader_election.cc:302] T 00000000000000000000000000000000 P 7234a0e1e66141dbb6e4535d5d68bb33 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7234a0e1e66141dbb6e4535d5d68bb33; no voters:
I20260808 16:56:05.792064 8930 leader_election.cc:288] T 00000000000000000000000000000000 P 7234a0e1e66141dbb6e4535d5d68bb33 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 16:56:05.792088 8937 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7234a0e1e66141dbb6e4535d5d68bb33 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:56:05.792187 8937 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7234a0e1e66141dbb6e4535d5d68bb33 [term 1 LEADER]: Becoming Leader. State: Replica: 7234a0e1e66141dbb6e4535d5d68bb33, State: Running, Role: LEADER
I20260808 16:56:05.792268 8937 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7234a0e1e66141dbb6e4535d5d68bb33 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7234a0e1e66141dbb6e4535d5d68bb33" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35831 } }
I20260808 16:56:05.792320 8930 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7234a0e1e66141dbb6e4535d5d68bb33 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 16:56:05.792570 8937 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7234a0e1e66141dbb6e4535d5d68bb33 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7234a0e1e66141dbb6e4535d5d68bb33. Latest consensus state: current_term: 1 leader_uuid: "7234a0e1e66141dbb6e4535d5d68bb33" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7234a0e1e66141dbb6e4535d5d68bb33" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35831 } } }
I20260808 16:56:05.792646 8937 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7234a0e1e66141dbb6e4535d5d68bb33 [sys.catalog]: This master's current role is: LEADER
I20260808 16:56:05.792583 8938 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7234a0e1e66141dbb6e4535d5d68bb33 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7234a0e1e66141dbb6e4535d5d68bb33" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7234a0e1e66141dbb6e4535d5d68bb33" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35831 } } }
I20260808 16:56:05.792690 8938 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7234a0e1e66141dbb6e4535d5d68bb33 [sys.catalog]: This master's current role is: LEADER
I20260808 16:56:05.793510 8944 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 16:56:05.794224 8944 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 08 16:56:05 dist-test-slave-6zbq krb5kdc[8811](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208165, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 16:56:05.796106 8944 catalog_manager.cc:1347] Generated new cluster ID: e3ed6e260eea4f00b38f7fcc317e0b9c
I20260808 16:56:05.796149 8944 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 16:56:05.799844 8935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:05.789282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.181:39679 (local address 127.0.0.1:35831)
0808 16:56:05.789436 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:05.789444 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:05.789468 (+ 24us) server_negotiation.cc:407] Connection header received
0808 16:56:05.789978 (+ 510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:05.789984 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:05.790031 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:05.790139 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:05.790940 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:05.792246 (+ 1306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:05.793289 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:05.793933 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:05.794040 (+ 107us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:05.796444 (+ 2404us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:05.796447 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:05.796448 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:56:05.796458 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:05.798248 (+ 1790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:05.798692 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:05.798695 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:05.798696 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:05.798800 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:05.799014 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:05.799014 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:05.799015 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:05.799281 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:05.799418 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:05.799554 (+ 136us) server_negotiation.cc:299] Negotiation successful
0808 16:56:05.799647 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":67,"threads_started":1}
I20260808 16:56:05.800740 8288 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 16:56:05.808944 8935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:05.802248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.181:48285 (local address 127.0.0.1:35831)
0808 16:56:05.802281 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:05.802282 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:05.802290 (+ 8us) server_negotiation.cc:407] Connection header received
0808 16:56:05.802308 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:05.802309 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:05.802318 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:05.802410 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:05.802968 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:05.803685 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:05.804595 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:05.804905 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:05.804963 (+ 58us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:05.806196 (+ 1233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:05.806197 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:05.806197 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:56:05.806204 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:05.807649 (+ 1445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:05.808046 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:05.808047 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:05.808047 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:05.808103 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:05.808302 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:05.808302 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:05.808302 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:05.808421 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:05.808518 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:05.808632 (+ 114us) server_negotiation.cc:299] Negotiation successful
0808 16:56:05.808693 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 16:56:05.809175 8944 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 16:56:05.809689 8944 catalog_manager.cc:1504] Loading token signing keys...
I20260808 16:56:05.813148 8944 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7234a0e1e66141dbb6e4535d5d68bb33: Generated new TSK 0
I20260808 16:56:05.813306 8944 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 16:56:05.815662 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 8831
W20260808 16:56:05.829121 8809 connection.cc:535] client connection to 127.0.0.1:35831 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35831): Connection reset by peer (error 104)
2026-08-08T16:56:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1399 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786207919384338-8288-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 16:56:05 dist-test-slave-6zbq krb5kdc[8982](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 16:56:05 dist-test-slave-6zbq krb5kdc[8982](info): set up 2 sockets
Aug 08 16:56:05 dist-test-slave-6zbq krb5kdc[8982](info): commencing operation
krb5kdc: starting...
W20260808 16:56:06.944958 8288 mini_kdc.cc:121] Time spent starting KDC: real 1.102s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786207919384338-8288-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786207919384338-8288-0/krb5kdc/test-user.keytab.
Aug 08 16:56:07 dist-test-slave-6zbq krb5kdc[8982](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208167, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T16:56:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T16:56:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786207919384338-8288-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786207919384338-8288-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 16:56:07.050819 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786207919384338-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786207919384338-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786207919384338-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38779
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786207919384338-8288-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38779
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:56:07.128937 8998 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:56:07.129118 8998 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:56:07.129140 8998 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:56:07.130257 8998 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 16:56:07.130299 8998 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:56:07.130312 8998 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:56:07.130326 8998 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 16:56:07.130371 8998 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 16:56:07.131806 8998 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786207919384338-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786207919384338-8288-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38779
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786207919384338-8288-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38779
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8998
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786207919384338-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:56:07.132012 8998 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:56:07.132200 8998 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:56:07.136540 9012 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:07.136554 9010 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:07.136551 9008 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:56:07.136662 8998 server_base.cc:1034] running on GCE node
I20260808 16:56:07.136957 8998 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:56:07.137210 8998 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:56:07.138356 8998 hybrid_clock.cc:648] HybridClock initialized: now 1786208167138328 us; error 26 us; skew 500 ppm
Aug 08 16:56:07 dist-test-slave-6zbq krb5kdc[8982](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208167, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:56:07.140558 8998 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 16:56:07.140960 8998 webserver.cc:460] Webserver started at http://127.0.0.1:33473/ using document root <none> and password file <none>
I20260808 16:56:07.141158 8998 fs_manager.cc:359] Metadata directory not provided
I20260808 16:56:07.141204 8998 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:56:07.141331 8998 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:56:07.142282 8998 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786207919384338-8288-0/minicluster-data/master-0/data/instance:
uuid: "f2a7242c3fa84fa48e89baa6348bc6a8"
format_stamp: "Formatted at 2026-08-08 16:56:07 on dist-test-slave-6zbq"
server_key: "790eec6a25f51161fe87691dcbec0cf7"
server_key_iv: "ebe03c258a25d46d4186fba7a9be1b72"
server_key_version: "encryptionkey@0"
I20260808 16:56:07.142642 8998 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786207919384338-8288-0/minicluster-data/master-0/wal/instance:
uuid: "f2a7242c3fa84fa48e89baa6348bc6a8"
format_stamp: "Formatted at 2026-08-08 16:56:07 on dist-test-slave-6zbq"
server_key: "790eec6a25f51161fe87691dcbec0cf7"
server_key_iv: "ebe03c258a25d46d4186fba7a9be1b72"
server_key_version: "encryptionkey@0"
I20260808 16:56:07.143812 8998 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 16:56:07.144443 9021 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:07.144601 8998 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 16:56:07.144687 8998 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786207919384338-8288-0/minicluster-data/master-0/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786207919384338-8288-0/minicluster-data/master-0/wal
uuid: "f2a7242c3fa84fa48e89baa6348bc6a8"
format_stamp: "Formatted at 2026-08-08 16:56:07 on dist-test-slave-6zbq"
server_key: "790eec6a25f51161fe87691dcbec0cf7"
server_key_iv: "ebe03c258a25d46d4186fba7a9be1b72"
server_key_version: "encryptionkey@0"
I20260808 16:56:07.144752 8998 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786207919384338-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786207919384338-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786207919384338-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:56:07.158994 8998 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:56:07.160866 8998 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:56:07.160995 8998 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:56:07.164619 8998 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38779
I20260808 16:56:07.164657 9098 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38779 every 8 connection(s)
I20260808 16:56:07.164953 8998 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
I20260808 16:56:07.165798 9099 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:56:07.167883 9099 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2a7242c3fa84fa48e89baa6348bc6a8: Bootstrap starting.
I20260808 16:56:07.168399 9099 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f2a7242c3fa84fa48e89baa6348bc6a8: Neither blocks nor log segments found. Creating new log.
I20260808 16:56:07.168642 9099 log.cc:824] T 00000000000000000000000000000000 P f2a7242c3fa84fa48e89baa6348bc6a8: Log is configured to *not* fsync() on all Append() calls
I20260808 16:56:07.169239 9099 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2a7242c3fa84fa48e89baa6348bc6a8: No bootstrap required, opened a new log
I20260808 16:56:07.170773 9099 raft_consensus.cc:348] T 00000000000000000000000000000000 P f2a7242c3fa84fa48e89baa6348bc6a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2a7242c3fa84fa48e89baa6348bc6a8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38779 } }
I20260808 16:56:07.170897 9099 raft_consensus.cc:374] T 00000000000000000000000000000000 P f2a7242c3fa84fa48e89baa6348bc6a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:56:07.170918 9099 raft_consensus.cc:729] T 00000000000000000000000000000000 P f2a7242c3fa84fa48e89baa6348bc6a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2a7242c3fa84fa48e89baa6348bc6a8, State: Initialized, Role: FOLLOWER
I20260808 16:56:07.171043 9099 consensus_queue.cc:260] T 00000000000000000000000000000000 P f2a7242c3fa84fa48e89baa6348bc6a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2a7242c3fa84fa48e89baa6348bc6a8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38779 } }
I20260808 16:56:07.171101 9099 raft_consensus.cc:388] T 00000000000000000000000000000000 P f2a7242c3fa84fa48e89baa6348bc6a8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 16:56:07.171123 9099 raft_consensus.cc:482] T 00000000000000000000000000000000 P f2a7242c3fa84fa48e89baa6348bc6a8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 16:56:07.171176 9099 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f2a7242c3fa84fa48e89baa6348bc6a8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:56:07.171630 9099 raft_consensus.cc:504] T 00000000000000000000000000000000 P f2a7242c3fa84fa48e89baa6348bc6a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2a7242c3fa84fa48e89baa6348bc6a8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38779 } }
I20260808 16:56:07.171720 9099 leader_election.cc:302] T 00000000000000000000000000000000 P f2a7242c3fa84fa48e89baa6348bc6a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f2a7242c3fa84fa48e89baa6348bc6a8; no voters:
I20260808 16:56:07.171859 9099 leader_election.cc:288] T 00000000000000000000000000000000 P f2a7242c3fa84fa48e89baa6348bc6a8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 16:56:07.171892 9102 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f2a7242c3fa84fa48e89baa6348bc6a8 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:56:07.172034 9102 raft_consensus.cc:686] T 00000000000000000000000000000000 P f2a7242c3fa84fa48e89baa6348bc6a8 [term 1 LEADER]: Becoming Leader. State: Replica: f2a7242c3fa84fa48e89baa6348bc6a8, State: Running, Role: LEADER
I20260808 16:56:07.172077 9099 sys_catalog.cc:564] T 00000000000000000000000000000000 P f2a7242c3fa84fa48e89baa6348bc6a8 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 16:56:07.172122 9102 consensus_queue.cc:237] T 00000000000000000000000000000000 P f2a7242c3fa84fa48e89baa6348bc6a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2a7242c3fa84fa48e89baa6348bc6a8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38779 } }
I20260808 16:56:07.172474 9104 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2a7242c3fa84fa48e89baa6348bc6a8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f2a7242c3fa84fa48e89baa6348bc6a8. Latest consensus state: current_term: 1 leader_uuid: "f2a7242c3fa84fa48e89baa6348bc6a8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2a7242c3fa84fa48e89baa6348bc6a8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38779 } } }
I20260808 16:56:07.172555 9104 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2a7242c3fa84fa48e89baa6348bc6a8 [sys.catalog]: This master's current role is: LEADER
I20260808 16:56:07.172678 9103 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2a7242c3fa84fa48e89baa6348bc6a8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f2a7242c3fa84fa48e89baa6348bc6a8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2a7242c3fa84fa48e89baa6348bc6a8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38779 } } }
I20260808 16:56:07.172789 9103 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2a7242c3fa84fa48e89baa6348bc6a8 [sys.catalog]: This master's current role is: LEADER
I20260808 16:56:07.172986 9113 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 16:56:07.173441 9113 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 16:56:07.174808 9113 catalog_manager.cc:1347] Generated new cluster ID: 1ec8d18cdf7448f3a140769fee2376a1
I20260808 16:56:07.174860 9113 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 16:56:07.174999 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 8998
I20260808 16:56:07.175104 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786207919384338-8288-0/minicluster-data/master-0/wal/instance
I20260808 16:56:07.175302 8288 external_mini_cluster.cc:1428] Setting key 5324c6400fdf3b4bd4ad4337e1c626dd
I20260808 16:56:07.179749 9131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:07.175658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.181:42793 (local address 127.0.0.1:38779)
0808 16:56:07.175946 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:07.175953 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:07.175974 (+ 21us) server_negotiation.cc:407] Connection header received
0808 16:56:07.176063 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:07.176068 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:07.176107 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:07.176212 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:07.176426 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:07.177301 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:07.178052 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:07.178483 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:07.178572 (+ 89us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:07.179142 (+ 570us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:07.179146 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:07.179146 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 16:56:07.179305 (+ 159us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:07.179333 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 16:56:07.179343 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:07.179413 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:07.179502 (+ 89us) server_negotiation.cc:299] Negotiation successful
0808 16:56:07.179604 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":238,"thread_start_us":66,"threads_started":1}
I20260808 16:56:07.180501 8288 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 16:56:07.184182 9131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:07.181806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.181:54485 (local address 127.0.0.1:38779)
0808 16:56:07.181839 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:07.181840 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:07.181847 (+ 7us) server_negotiation.cc:407] Connection header received
0808 16:56:07.181866 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:07.181867 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:07.181877 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:07.181922 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:07.182045 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:07.182543 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:07.183178 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:07.183390 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:07.183443 (+ 53us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:07.183964 (+ 521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:07.183965 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:07.183966 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 16:56:07.183970 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:07.183983 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 16:56:07.183987 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:07.184035 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:07.184100 (+ 65us) server_negotiation.cc:299] Negotiation successful
0808 16:56:07.184111 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 16:56:07.186509 9113 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 16:56:07.187160 9113 catalog_manager.cc:1504] Loading token signing keys...
I20260808 16:56:07.189893 9113 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f2a7242c3fa84fa48e89baa6348bc6a8: Generated new TSK 0
I20260808 16:56:07.190035 9113 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 16:56:07.193732 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 8998
W20260808 16:56:07.214309 8977 connection.cc:535] client connection to 127.0.0.1:38779 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38779): Connection reset by peer (error 104)
2026-08-08T16:56:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1385 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786207919384338-8288-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 16:56:07 dist-test-slave-6zbq krb5kdc[9141](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 16:56:07 dist-test-slave-6zbq krb5kdc[9141](info): set up 2 sockets
Aug 08 16:56:07 dist-test-slave-6zbq krb5kdc[9141](info): commencing operation
krb5kdc: starting...
W20260808 16:56:08.326365 8288 mini_kdc.cc:121] Time spent starting KDC: real 1.099s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786207919384338-8288-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786207919384338-8288-0/krb5kdc/test-user.keytab.
Aug 08 16:56:08 dist-test-slave-6zbq krb5kdc[9141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208168, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T16:56:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T16:56:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786207919384338-8288-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786207919384338-8288-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 16:56:08.430043 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786207919384338-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786207919384338-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786207919384338-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46693
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786207919384338-8288-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46693
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:56:08.508105 9164 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:56:08.508291 9164 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:56:08.508312 9164 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:56:08.509354 9164 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 16:56:08.509387 9164 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:56:08.509399 9164 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:56:08.509410 9164 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 16:56:08.509424 9164 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 16:56:08.510804 9164 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786207919384338-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786207919384338-8288-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46693
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786207919384338-8288-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46693
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9164
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786207919384338-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:56:08.510982 9164 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:56:08.511147 9164 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:56:08.515796 9173 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:56:08.515826 9164 server_base.cc:1034] running on GCE node
W20260808 16:56:08.515868 9171 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:08.515794 9176 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:56:08.516253 9164 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:56:08.516532 9164 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:56:08.517686 9164 hybrid_clock.cc:648] HybridClock initialized: now 1786208168517668 us; error 33 us; skew 500 ppm
Aug 08 16:56:08 dist-test-slave-6zbq krb5kdc[9141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208168, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:56:08.520085 9164 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 16:56:08.520481 9164 webserver.cc:460] Webserver started at http://127.0.0.1:41481/ using document root <none> and password file <none>
I20260808 16:56:08.520676 9164 fs_manager.cc:359] Metadata directory not provided
I20260808 16:56:08.520721 9164 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:56:08.520850 9164 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:56:08.521813 9164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786207919384338-8288-0/minicluster-data/master-0/data/instance:
uuid: "25b2e63ce78e4ec1a06d0057dba3f3d7"
format_stamp: "Formatted at 2026-08-08 16:56:08 on dist-test-slave-6zbq"
server_key: "6d08ed0402c8454b589effc6c8b5d25f"
server_key_iv: "947230a8671090b3fac58402350039c1"
server_key_version: "encryptionkey@0"
I20260808 16:56:08.522126 9164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786207919384338-8288-0/minicluster-data/master-0/wal/instance:
uuid: "25b2e63ce78e4ec1a06d0057dba3f3d7"
format_stamp: "Formatted at 2026-08-08 16:56:08 on dist-test-slave-6zbq"
server_key: "6d08ed0402c8454b589effc6c8b5d25f"
server_key_iv: "947230a8671090b3fac58402350039c1"
server_key_version: "encryptionkey@0"
I20260808 16:56:08.523375 9164 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 16:56:08.524026 9189 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:08.524171 9164 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 16:56:08.524252 9164 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786207919384338-8288-0/minicluster-data/master-0/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786207919384338-8288-0/minicluster-data/master-0/wal
uuid: "25b2e63ce78e4ec1a06d0057dba3f3d7"
format_stamp: "Formatted at 2026-08-08 16:56:08 on dist-test-slave-6zbq"
server_key: "6d08ed0402c8454b589effc6c8b5d25f"
server_key_iv: "947230a8671090b3fac58402350039c1"
server_key_version: "encryptionkey@0"
I20260808 16:56:08.524317 9164 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786207919384338-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786207919384338-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786207919384338-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:56:08.527417 9164 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:56:08.529323 9164 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:56:08.529445 9164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:56:08.533018 9164 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46693
I20260808 16:56:08.533097 9256 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46693 every 8 connection(s)
I20260808 16:56:08.533329 9164 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
I20260808 16:56:08.533555 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 9164
I20260808 16:56:08.533686 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786207919384338-8288-0/minicluster-data/master-0/wal/instance
I20260808 16:56:08.533879 8288 external_mini_cluster.cc:1428] Setting key 4722c72e28e26f6172b4d5ece29ff875
I20260808 16:56:08.534433 9258 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:56:08.535993 9264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:08.534333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.181:45895 (local address 127.0.0.1:46693)
0808 16:56:08.534523 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:08.534532 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:08.534562 (+ 30us) server_negotiation.cc:407] Connection header received
0808 16:56:08.534941 (+ 379us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:08.534947 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:08.534970 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:08.535073 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:08.535611 (+ 538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:08.535614 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:08.535614 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 16:56:08.535618 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:08.535650 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 16:56:08.535659 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:08.535711 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:08.535789 (+ 78us) server_negotiation.cc:299] Negotiation successful
0808 16:56:08.535822 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":68,"threads_started":1}
I20260808 16:56:08.536854 8288 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 16:56:08.537086 9258 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25b2e63ce78e4ec1a06d0057dba3f3d7: Bootstrap starting.
I20260808 16:56:08.537737 9258 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 25b2e63ce78e4ec1a06d0057dba3f3d7: Neither blocks nor log segments found. Creating new log.
I20260808 16:56:08.538064 9258 log.cc:824] T 00000000000000000000000000000000 P 25b2e63ce78e4ec1a06d0057dba3f3d7: Log is configured to *not* fsync() on all Append() calls
I20260808 16:56:08.538822 9258 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25b2e63ce78e4ec1a06d0057dba3f3d7: No bootstrap required, opened a new log
I20260808 16:56:08.539259 9264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:08.538291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.181:40833 (local address 127.0.0.1:46693)
0808 16:56:08.538359 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:08.538361 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:08.538477 (+ 116us) server_negotiation.cc:407] Connection header received
0808 16:56:08.538540 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:08.538541 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:08.538545 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:08.538605 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:08.539077 (+ 472us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:08.539078 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:08.539078 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 16:56:08.539082 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:08.539095 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 16:56:08.539099 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:08.539157 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:08.539191 (+ 34us) server_negotiation.cc:299] Negotiation successful
0808 16:56:08.539201 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 16:56:08.540700 9258 raft_consensus.cc:348] T 00000000000000000000000000000000 P 25b2e63ce78e4ec1a06d0057dba3f3d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25b2e63ce78e4ec1a06d0057dba3f3d7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46693 } }
I20260808 16:56:08.540825 9258 raft_consensus.cc:374] T 00000000000000000000000000000000 P 25b2e63ce78e4ec1a06d0057dba3f3d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:56:08.540848 9258 raft_consensus.cc:729] T 00000000000000000000000000000000 P 25b2e63ce78e4ec1a06d0057dba3f3d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25b2e63ce78e4ec1a06d0057dba3f3d7, State: Initialized, Role: FOLLOWER
I20260808 16:56:08.540937 9258 consensus_queue.cc:260] T 00000000000000000000000000000000 P 25b2e63ce78e4ec1a06d0057dba3f3d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25b2e63ce78e4ec1a06d0057dba3f3d7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46693 } }
I20260808 16:56:08.541025 9258 raft_consensus.cc:388] T 00000000000000000000000000000000 P 25b2e63ce78e4ec1a06d0057dba3f3d7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 16:56:08.541052 9258 raft_consensus.cc:482] T 00000000000000000000000000000000 P 25b2e63ce78e4ec1a06d0057dba3f3d7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 16:56:08.541082 9258 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 25b2e63ce78e4ec1a06d0057dba3f3d7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:56:08.541543 9258 raft_consensus.cc:504] T 00000000000000000000000000000000 P 25b2e63ce78e4ec1a06d0057dba3f3d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25b2e63ce78e4ec1a06d0057dba3f3d7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46693 } }
I20260808 16:56:08.541625 9258 leader_election.cc:302] T 00000000000000000000000000000000 P 25b2e63ce78e4ec1a06d0057dba3f3d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 25b2e63ce78e4ec1a06d0057dba3f3d7; no voters:
I20260808 16:56:08.541750 9258 leader_election.cc:288] T 00000000000000000000000000000000 P 25b2e63ce78e4ec1a06d0057dba3f3d7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 16:56:08.541795 9278 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 25b2e63ce78e4ec1a06d0057dba3f3d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:56:08.541942 9258 sys_catalog.cc:564] T 00000000000000000000000000000000 P 25b2e63ce78e4ec1a06d0057dba3f3d7 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 16:56:08.541997 9278 raft_consensus.cc:686] T 00000000000000000000000000000000 P 25b2e63ce78e4ec1a06d0057dba3f3d7 [term 1 LEADER]: Becoming Leader. State: Replica: 25b2e63ce78e4ec1a06d0057dba3f3d7, State: Running, Role: LEADER
I20260808 16:56:08.542119 9278 consensus_queue.cc:237] T 00000000000000000000000000000000 P 25b2e63ce78e4ec1a06d0057dba3f3d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25b2e63ce78e4ec1a06d0057dba3f3d7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46693 } }
I20260808 16:56:08.542481 9279 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25b2e63ce78e4ec1a06d0057dba3f3d7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "25b2e63ce78e4ec1a06d0057dba3f3d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25b2e63ce78e4ec1a06d0057dba3f3d7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46693 } } }
I20260808 16:56:08.542573 9279 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25b2e63ce78e4ec1a06d0057dba3f3d7 [sys.catalog]: This master's current role is: LEADER
I20260808 16:56:08.542805 9280 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25b2e63ce78e4ec1a06d0057dba3f3d7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 25b2e63ce78e4ec1a06d0057dba3f3d7. Latest consensus state: current_term: 1 leader_uuid: "25b2e63ce78e4ec1a06d0057dba3f3d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25b2e63ce78e4ec1a06d0057dba3f3d7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46693 } } }
I20260808 16:56:08.542806 9289 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 16:56:08.542951 9280 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25b2e63ce78e4ec1a06d0057dba3f3d7 [sys.catalog]: This master's current role is: LEADER
I20260808 16:56:08.543424 9289 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 16:56:08.544709 9289 catalog_manager.cc:1347] Generated new cluster ID: e357fda937a24eddacdb4fcb84d2b008
I20260808 16:56:08.544759 9289 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 16:56:08.550431 9289 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 16:56:08.550570 9289 catalog_manager.cc:1504] Loading token signing keys...
I20260808 16:56:08.553881 9289 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 25b2e63ce78e4ec1a06d0057dba3f3d7: Generated new TSK 0
I20260808 16:56:08.554028 9289 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 16:56:08.559429 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 9164
2026-08-08T16:56:08Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1355 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786207919384338-8288-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 16:56:08 dist-test-slave-6zbq krb5kdc[9310](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 16:56:08 dist-test-slave-6zbq krb5kdc[9310](info): set up 2 sockets
Aug 08 16:56:08 dist-test-slave-6zbq krb5kdc[9310](info): commencing operation
krb5kdc: starting...
W20260808 16:56:09.684031 8288 mini_kdc.cc:121] Time spent starting KDC: real 1.101s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786207919384338-8288-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786207919384338-8288-0/krb5kdc/test-user.keytab.
Aug 08 16:56:09 dist-test-slave-6zbq krb5kdc[9310](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208169, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T16:56:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T16:56:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786207919384338-8288-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786207919384338-8288-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 16:56:09.790875 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786207919384338-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786207919384338-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786207919384338-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41119
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786207919384338-8288-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41119
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:56:09.869836 9327 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:56:09.870033 9327 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:56:09.870064 9327 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:56:09.871199 9327 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 16:56:09.871239 9327 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:56:09.871258 9327 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:56:09.871274 9327 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 16:56:09.871289 9327 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 16:56:09.872723 9327 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786207919384338-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786207919384338-8288-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41119
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786207919384338-8288-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41119
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9327
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786207919384338-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:56:09.872951 9327 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:56:09.873145 9327 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:56:09.877528 9333 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:56:09.877588 9327 server_base.cc:1034] running on GCE node
W20260808 16:56:09.877544 9336 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:09.877537 9342 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:56:09.877970 9327 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:56:09.878228 9327 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:56:09.879364 9327 hybrid_clock.cc:648] HybridClock initialized: now 1786208169879350 us; error 31 us; skew 500 ppm
Aug 08 16:56:09 dist-test-slave-6zbq krb5kdc[9310](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208169, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:56:09.881603 9327 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 16:56:09.881994 9327 webserver.cc:460] Webserver started at http://127.0.0.1:46871/ using document root <none> and password file <none>
I20260808 16:56:09.882217 9327 fs_manager.cc:359] Metadata directory not provided
I20260808 16:56:09.882264 9327 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:56:09.882411 9327 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:56:09.883383 9327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786207919384338-8288-0/minicluster-data/master-0/data/instance:
uuid: "d2e3317f0a3d4b25897b578619638855"
format_stamp: "Formatted at 2026-08-08 16:56:09 on dist-test-slave-6zbq"
server_key: "54fbfbef873c553acd8e1f1dd052e683"
server_key_iv: "2896bad113c00731d3e8cfc2124e69b2"
server_key_version: "encryptionkey@0"
I20260808 16:56:09.883713 9327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786207919384338-8288-0/minicluster-data/master-0/wal/instance:
uuid: "d2e3317f0a3d4b25897b578619638855"
format_stamp: "Formatted at 2026-08-08 16:56:09 on dist-test-slave-6zbq"
server_key: "54fbfbef873c553acd8e1f1dd052e683"
server_key_iv: "2896bad113c00731d3e8cfc2124e69b2"
server_key_version: "encryptionkey@0"
I20260808 16:56:09.884950 9327 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 16:56:09.885560 9351 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:09.885699 9327 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 16:56:09.885772 9327 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786207919384338-8288-0/minicluster-data/master-0/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786207919384338-8288-0/minicluster-data/master-0/wal
uuid: "d2e3317f0a3d4b25897b578619638855"
format_stamp: "Formatted at 2026-08-08 16:56:09 on dist-test-slave-6zbq"
server_key: "54fbfbef873c553acd8e1f1dd052e683"
server_key_iv: "2896bad113c00731d3e8cfc2124e69b2"
server_key_version: "encryptionkey@0"
I20260808 16:56:09.885833 9327 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786207919384338-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786207919384338-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786207919384338-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:56:09.901870 9327 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:56:09.903465 9327 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:56:09.903584 9327 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:56:09.907203 9327 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41119
I20260808 16:56:09.907234 9425 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41119 every 8 connection(s)
I20260808 16:56:09.907531 9327 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
I20260808 16:56:09.908406 9426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:56:09.910828 9426 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2e3317f0a3d4b25897b578619638855: Bootstrap starting.
I20260808 16:56:09.911350 9426 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d2e3317f0a3d4b25897b578619638855: Neither blocks nor log segments found. Creating new log.
I20260808 16:56:09.911593 9426 log.cc:824] T 00000000000000000000000000000000 P d2e3317f0a3d4b25897b578619638855: Log is configured to *not* fsync() on all Append() calls
I20260808 16:56:09.912195 9426 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2e3317f0a3d4b25897b578619638855: No bootstrap required, opened a new log
I20260808 16:56:09.913679 9426 raft_consensus.cc:348] T 00000000000000000000000000000000 P d2e3317f0a3d4b25897b578619638855 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2e3317f0a3d4b25897b578619638855" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41119 } }
I20260808 16:56:09.913802 9426 raft_consensus.cc:374] T 00000000000000000000000000000000 P d2e3317f0a3d4b25897b578619638855 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:56:09.913834 9426 raft_consensus.cc:729] T 00000000000000000000000000000000 P d2e3317f0a3d4b25897b578619638855 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2e3317f0a3d4b25897b578619638855, State: Initialized, Role: FOLLOWER
I20260808 16:56:09.913960 9426 consensus_queue.cc:260] T 00000000000000000000000000000000 P d2e3317f0a3d4b25897b578619638855 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2e3317f0a3d4b25897b578619638855" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41119 } }
I20260808 16:56:09.914034 9426 raft_consensus.cc:388] T 00000000000000000000000000000000 P d2e3317f0a3d4b25897b578619638855 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 16:56:09.914065 9426 raft_consensus.cc:482] T 00000000000000000000000000000000 P d2e3317f0a3d4b25897b578619638855 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 16:56:09.914111 9426 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d2e3317f0a3d4b25897b578619638855 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:56:09.914623 9426 raft_consensus.cc:504] T 00000000000000000000000000000000 P d2e3317f0a3d4b25897b578619638855 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2e3317f0a3d4b25897b578619638855" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41119 } }
I20260808 16:56:09.914716 9426 leader_election.cc:302] T 00000000000000000000000000000000 P d2e3317f0a3d4b25897b578619638855 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d2e3317f0a3d4b25897b578619638855; no voters:
I20260808 16:56:09.914799 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 9327
I20260808 16:56:09.914861 9426 leader_election.cc:288] T 00000000000000000000000000000000 P d2e3317f0a3d4b25897b578619638855 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 16:56:09.914909 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786207919384338-8288-0/minicluster-data/master-0/wal/instance
I20260808 16:56:09.914907 9430 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d2e3317f0a3d4b25897b578619638855 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:56:09.915016 9430 raft_consensus.cc:686] T 00000000000000000000000000000000 P d2e3317f0a3d4b25897b578619638855 [term 1 LEADER]: Becoming Leader. State: Replica: d2e3317f0a3d4b25897b578619638855, State: Running, Role: LEADER
I20260808 16:56:09.915105 8288 external_mini_cluster.cc:1428] Setting key 7ed1d1c5ad167f10e7a43537fa78cca9
I20260808 16:56:09.915134 9426 sys_catalog.cc:564] T 00000000000000000000000000000000 P d2e3317f0a3d4b25897b578619638855 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 16:56:09.915153 9430 consensus_queue.cc:237] T 00000000000000000000000000000000 P d2e3317f0a3d4b25897b578619638855 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2e3317f0a3d4b25897b578619638855" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41119 } }
I20260808 16:56:09.915663 9432 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2e3317f0a3d4b25897b578619638855 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d2e3317f0a3d4b25897b578619638855" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2e3317f0a3d4b25897b578619638855" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41119 } } }
I20260808 16:56:09.915748 9432 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2e3317f0a3d4b25897b578619638855 [sys.catalog]: This master's current role is: LEADER
I20260808 16:56:09.915966 9437 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 16:56:09.916003 9433 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2e3317f0a3d4b25897b578619638855 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d2e3317f0a3d4b25897b578619638855. Latest consensus state: current_term: 1 leader_uuid: "d2e3317f0a3d4b25897b578619638855" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2e3317f0a3d4b25897b578619638855" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41119 } } }
I20260808 16:56:09.916076 9433 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2e3317f0a3d4b25897b578619638855 [sys.catalog]: This master's current role is: LEADER
I20260808 16:56:09.916496 9437 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 16:56:09.917982 9437 catalog_manager.cc:1347] Generated new cluster ID: 06cb4d0e50664a22aeb7f44fd075a462
I20260808 16:56:09.918035 9437 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 16:56:09 dist-test-slave-6zbq krb5kdc[9310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208169, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 16:56:09.925201 9439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:09.915511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.181:54579 (local address 127.0.0.1:41119)
0808 16:56:09.917115 (+ 1604us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:09.917124 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:09.917159 (+ 35us) server_negotiation.cc:407] Connection header received
0808 16:56:09.917266 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:09.917272 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:09.917321 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:09.917441 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:09.918124 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:09.919032 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:09.919762 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:09.920197 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:09.920289 (+ 92us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:09.922396 (+ 2107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:09.922400 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:09.922400 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:56:09.922411 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:09.923816 (+ 1405us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:09.924178 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:09.924180 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:09.924181 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:09.924270 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:09.924470 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:09.924471 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:09.924471 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:09.924676 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:09.924826 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:09.924953 (+ 127us) server_negotiation.cc:299] Negotiation successful
0808 16:56:09.925046 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1562,"thread_start_us":484,"threads_started":1}
I20260808 16:56:09.925995 8288 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 16:56:09.935936 9439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:09.930631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.181:57453 (local address 127.0.0.1:41119)
0808 16:56:09.930664 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:09.930666 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:09.930818 (+ 152us) server_negotiation.cc:407] Connection header received
0808 16:56:09.930853 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:09.930854 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:09.930863 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:09.930906 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:09.931382 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:09.931844 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:09.932474 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:09.932667 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:09.932718 (+ 51us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:09.933722 (+ 1004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:09.933724 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:09.933724 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:56:09.933731 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:09.934936 (+ 1205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:09.935258 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:09.935259 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:09.935259 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:09.935309 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:09.935490 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:09.935491 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:09.935491 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:09.935585 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:09.935663 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:09.935805 (+ 142us) server_negotiation.cc:299] Negotiation successful
0808 16:56:09.935860 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260808 16:56:09.941298 9437 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 16:56:09.941965 9437 catalog_manager.cc:1504] Loading token signing keys...
I20260808 16:56:09.955072 9437 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d2e3317f0a3d4b25897b578619638855: Generated new TSK 0
I20260808 16:56:09.955224 9437 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 16:56:09.958451 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 9327
W20260808 16:56:09.972651 9306 connection.cc:535] client connection to 127.0.0.1:41119 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41119): Connection reset by peer (error 104)
2026-08-08T16:56:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1402 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786207919384338-8288-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 16:56:10 dist-test-slave-6zbq krb5kdc[9475](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 16:56:10 dist-test-slave-6zbq krb5kdc[9475](info): set up 2 sockets
Aug 08 16:56:10 dist-test-slave-6zbq krb5kdc[9475](info): commencing operation
krb5kdc: starting...
W20260808 16:56:11.118145 8288 mini_kdc.cc:121] Time spent starting KDC: real 1.132s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786207919384338-8288-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786207919384338-8288-0/krb5kdc/test-user.keytab.
Aug 08 16:56:11 dist-test-slave-6zbq krb5kdc[9475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208171, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T16:56:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T16:56:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786207919384338-8288-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786207919384338-8288-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 16:56:11.262393 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786207919384338-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786207919384338-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786207919384338-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42641
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786207919384338-8288-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42641
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:56:11.340768 9490 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:56:11.340946 9490 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:56:11.340966 9490 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:56:11.342038 9490 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 16:56:11.342073 9490 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:56:11.342085 9490 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:56:11.342096 9490 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 16:56:11.342118 9490 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 16:56:11.343531 9490 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786207919384338-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786207919384338-8288-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42641
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786207919384338-8288-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42641
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9490
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786207919384338-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:56:11.343711 9490 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:56:11.343879 9490 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:56:11.348222 9501 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:11.348270 9497 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:11.348232 9495 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:56:11.348467 9490 server_base.cc:1034] running on GCE node
I20260808 16:56:11.348686 9490 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:56:11.348943 9490 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:56:11.350072 9490 hybrid_clock.cc:648] HybridClock initialized: now 1786208171350062 us; error 30 us; skew 500 ppm
Aug 08 16:56:11 dist-test-slave-6zbq krb5kdc[9475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208171, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:56:11.357486 9490 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 16:56:11.357849 9490 webserver.cc:460] Webserver started at http://127.0.0.1:46665/ using document root <none> and password file <none>
I20260808 16:56:11.358031 9490 fs_manager.cc:359] Metadata directory not provided
I20260808 16:56:11.358067 9490 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:56:11.358151 9490 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:56:11.359083 9490 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786207919384338-8288-0/minicluster-data/master-0/data/instance:
uuid: "3656593be5e04b998b034de0f6a39633"
format_stamp: "Formatted at 2026-08-08 16:56:11 on dist-test-slave-6zbq"
server_key: "9a9ee4f790e305a3996ba170dee56daa"
server_key_iv: "f821439043411c960491675fd4f20a48"
server_key_version: "encryptionkey@0"
I20260808 16:56:11.359345 9490 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786207919384338-8288-0/minicluster-data/master-0/wal/instance:
uuid: "3656593be5e04b998b034de0f6a39633"
format_stamp: "Formatted at 2026-08-08 16:56:11 on dist-test-slave-6zbq"
server_key: "9a9ee4f790e305a3996ba170dee56daa"
server_key_iv: "f821439043411c960491675fd4f20a48"
server_key_version: "encryptionkey@0"
I20260808 16:56:11.360430 9490 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 16:56:11.361032 9509 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:11.361194 9490 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 16:56:11.361276 9490 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786207919384338-8288-0/minicluster-data/master-0/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786207919384338-8288-0/minicluster-data/master-0/wal
uuid: "3656593be5e04b998b034de0f6a39633"
format_stamp: "Formatted at 2026-08-08 16:56:11 on dist-test-slave-6zbq"
server_key: "9a9ee4f790e305a3996ba170dee56daa"
server_key_iv: "f821439043411c960491675fd4f20a48"
server_key_version: "encryptionkey@0"
I20260808 16:56:11.361328 9490 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786207919384338-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786207919384338-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786207919384338-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:56:11.373862 9490 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:56:11.375763 9490 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:56:11.375860 9490 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:56:11.379340 9490 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42641
I20260808 16:56:11.379388 9593 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42641 every 8 connection(s)
I20260808 16:56:11.379652 9490 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
I20260808 16:56:11.380398 9594 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:56:11.382464 9594 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3656593be5e04b998b034de0f6a39633: Bootstrap starting.
I20260808 16:56:11.382980 9594 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3656593be5e04b998b034de0f6a39633: Neither blocks nor log segments found. Creating new log.
I20260808 16:56:11.383222 9594 log.cc:824] T 00000000000000000000000000000000 P 3656593be5e04b998b034de0f6a39633: Log is configured to *not* fsync() on all Append() calls
I20260808 16:56:11.383818 9594 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3656593be5e04b998b034de0f6a39633: No bootstrap required, opened a new log
I20260808 16:56:11.385346 9594 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3656593be5e04b998b034de0f6a39633 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3656593be5e04b998b034de0f6a39633" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42641 } }
I20260808 16:56:11.385469 9594 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3656593be5e04b998b034de0f6a39633 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:56:11.385501 9594 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3656593be5e04b998b034de0f6a39633 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3656593be5e04b998b034de0f6a39633, State: Initialized, Role: FOLLOWER
I20260808 16:56:11.385628 9594 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3656593be5e04b998b034de0f6a39633 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3656593be5e04b998b034de0f6a39633" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42641 } }
I20260808 16:56:11.385697 9594 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3656593be5e04b998b034de0f6a39633 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 16:56:11.385730 9594 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3656593be5e04b998b034de0f6a39633 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 16:56:11.385773 9594 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3656593be5e04b998b034de0f6a39633 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:56:11.386252 9594 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3656593be5e04b998b034de0f6a39633 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3656593be5e04b998b034de0f6a39633" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42641 } }
I20260808 16:56:11.386379 9594 leader_election.cc:302] T 00000000000000000000000000000000 P 3656593be5e04b998b034de0f6a39633 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3656593be5e04b998b034de0f6a39633; no voters:
I20260808 16:56:11.386425 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 9490
I20260808 16:56:11.386543 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786207919384338-8288-0/minicluster-data/master-0/wal/instance
I20260808 16:56:11.386544 9594 leader_election.cc:288] T 00000000000000000000000000000000 P 3656593be5e04b998b034de0f6a39633 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 16:56:11.386569 9599 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3656593be5e04b998b034de0f6a39633 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:56:11.386731 8288 external_mini_cluster.cc:1428] Setting key b0b4ceddbac92f89b3418b5af4cf4780
I20260808 16:56:11.386752 9599 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3656593be5e04b998b034de0f6a39633 [term 1 LEADER]: Becoming Leader. State: Replica: 3656593be5e04b998b034de0f6a39633, State: Running, Role: LEADER
I20260808 16:56:11.386780 9594 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3656593be5e04b998b034de0f6a39633 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 16:56:11.386832 9599 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3656593be5e04b998b034de0f6a39633 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3656593be5e04b998b034de0f6a39633" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42641 } }
I20260808 16:56:11.387274 9600 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3656593be5e04b998b034de0f6a39633 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3656593be5e04b998b034de0f6a39633" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3656593be5e04b998b034de0f6a39633" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42641 } } }
I20260808 16:56:11.387351 9600 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3656593be5e04b998b034de0f6a39633 [sys.catalog]: This master's current role is: LEADER
I20260808 16:56:11.387552 9601 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3656593be5e04b998b034de0f6a39633 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3656593be5e04b998b034de0f6a39633. Latest consensus state: current_term: 1 leader_uuid: "3656593be5e04b998b034de0f6a39633" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3656593be5e04b998b034de0f6a39633" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42641 } } }
I20260808 16:56:11.387670 9601 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3656593be5e04b998b034de0f6a39633 [sys.catalog]: This master's current role is: LEADER
I20260808 16:56:11.387765 9614 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 16:56:11.388316 9614 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 16:56:11.389827 9614 catalog_manager.cc:1347] Generated new cluster ID: d1d06025587846dfb470c4636274ae63
I20260808 16:56:11.389878 9614 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 16:56:11.392606 9610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:11.387254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.181:47443 (local address 127.0.0.1:42641)
0808 16:56:11.387422 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:11.387430 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:11.387601 (+ 171us) server_negotiation.cc:407] Connection header received
0808 16:56:11.387827 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:11.387834 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:11.387896 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:11.388004 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:11.388357 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:11.389617 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:11.390684 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:11.391307 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:11.391428 (+ 121us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:11.391972 (+ 544us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:11.391976 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:11.391976 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 16:56:11.392168 (+ 192us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:11.392218 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 16:56:11.392232 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:11.392321 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:11.392384 (+ 63us) server_negotiation.cc:299] Negotiation successful
0808 16:56:11.392411 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":68,"threads_started":1}
I20260808 16:56:11.403275 8288 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 16:56:11.413473 9614 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 16:56:11.414136 9614 catalog_manager.cc:1504] Loading token signing keys...
I20260808 16:56:11.415799 9610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:11.404695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.181:60273 (local address 127.0.0.1:42641)
0808 16:56:11.404727 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:11.404729 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:11.413125 (+ 8396us) server_negotiation.cc:407] Connection header received
0808 16:56:11.413181 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:11.413182 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:11.413195 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:11.413249 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:11.413412 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:11.413936 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:11.414718 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:11.414939 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:11.414993 (+ 54us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:11.415570 (+ 577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:11.415571 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:11.415572 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 16:56:11.415576 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:11.415592 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 16:56:11.415597 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:11.415647 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:11.415715 (+ 68us) server_negotiation.cc:299] Negotiation successful
0808 16:56:11.415727 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260808 16:56:11.421316 9614 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3656593be5e04b998b034de0f6a39633: Generated new TSK 0
I20260808 16:56:11.421468 9614 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 16:56:11.426393 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 9490
W20260808 16:56:11.441489 9471 connection.cc:535] client connection to 127.0.0.1:42641 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42641): Connection reset by peer (error 104)
2026-08-08T16:56:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1468 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786207919384338-8288-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 16:56:11 dist-test-slave-6zbq krb5kdc[9645](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 16:56:11 dist-test-slave-6zbq krb5kdc[9645](info): set up 2 sockets
Aug 08 16:56:11 dist-test-slave-6zbq krb5kdc[9645](info): commencing operation
krb5kdc: starting...
W20260808 16:56:12.561124 8288 mini_kdc.cc:121] Time spent starting KDC: real 1.106s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786207919384338-8288-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786207919384338-8288-0/krb5kdc/test-user.keytab.
Aug 08 16:56:12 dist-test-slave-6zbq krb5kdc[9645](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208172, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T16:56:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T16:56:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786207919384338-8288-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786207919384338-8288-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 16:56:12.668934 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786207919384338-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786207919384338-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786207919384338-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43647
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786207919384338-8288-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43647
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:56:12.746016 9661 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:56:12.746202 9661 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:56:12.746232 9661 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:56:12.747340 9661 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 16:56:12.747382 9661 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:56:12.747401 9661 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:56:12.747418 9661 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 16:56:12.747433 9661 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 16:56:12.748855 9661 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786207919384338-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786207919384338-8288-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43647
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786207919384338-8288-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43647
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9661
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786207919384338-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:56:12.749085 9661 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:56:12.749289 9661 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:56:12.753609 9672 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:12.753643 9667 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:12.753666 9668 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:56:12.753865 9661 server_base.cc:1034] running on GCE node
I20260808 16:56:12.754086 9661 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:56:12.754408 9661 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:56:12.755553 9661 hybrid_clock.cc:648] HybridClock initialized: now 1786208172755535 us; error 27 us; skew 500 ppm
Aug 08 16:56:12 dist-test-slave-6zbq krb5kdc[9645](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208172, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:56:12.757903 9661 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 16:56:12.758304 9661 webserver.cc:460] Webserver started at http://127.0.0.1:43007/ using document root <none> and password file <none>
I20260808 16:56:12.758525 9661 fs_manager.cc:359] Metadata directory not provided
I20260808 16:56:12.758571 9661 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:56:12.758693 9661 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:56:12.759656 9661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786207919384338-8288-0/minicluster-data/master-0/data/instance:
uuid: "a7410dc5dbaf405281198230fc371f96"
format_stamp: "Formatted at 2026-08-08 16:56:12 on dist-test-slave-6zbq"
server_key: "a8d7f4c8b707e6c26b9b46e94c3071d7"
server_key_iv: "8096d0e61ebced72b55161da36d12a72"
server_key_version: "encryptionkey@0"
I20260808 16:56:12.759976 9661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786207919384338-8288-0/minicluster-data/master-0/wal/instance:
uuid: "a7410dc5dbaf405281198230fc371f96"
format_stamp: "Formatted at 2026-08-08 16:56:12 on dist-test-slave-6zbq"
server_key: "a8d7f4c8b707e6c26b9b46e94c3071d7"
server_key_iv: "8096d0e61ebced72b55161da36d12a72"
server_key_version: "encryptionkey@0"
I20260808 16:56:12.761149 9661 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 16:56:12.761786 9683 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:12.761931 9661 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:12.762012 9661 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786207919384338-8288-0/minicluster-data/master-0/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786207919384338-8288-0/minicluster-data/master-0/wal
uuid: "a7410dc5dbaf405281198230fc371f96"
format_stamp: "Formatted at 2026-08-08 16:56:12 on dist-test-slave-6zbq"
server_key: "a8d7f4c8b707e6c26b9b46e94c3071d7"
server_key_iv: "8096d0e61ebced72b55161da36d12a72"
server_key_version: "encryptionkey@0"
I20260808 16:56:12.762075 9661 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786207919384338-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786207919384338-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786207919384338-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:56:12.765120 9661 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:56:12.766716 9661 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:56:12.766837 9661 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:56:12.770293 9661 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43647
I20260808 16:56:12.770321 9754 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43647 every 8 connection(s)
I20260808 16:56:12.770624 9661 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
I20260808 16:56:12.771404 9755 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:56:12.772595 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 9661
I20260808 16:56:12.772692 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786207919384338-8288-0/minicluster-data/master-0/wal/instance
I20260808 16:56:12.772850 8288 external_mini_cluster.cc:1428] Setting key 82fddee29d2dcce841b16cc3661a5bfd
I20260808 16:56:12.773725 9755 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7410dc5dbaf405281198230fc371f96: Bootstrap starting.
I20260808 16:56:12.774417 9755 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a7410dc5dbaf405281198230fc371f96: Neither blocks nor log segments found. Creating new log.
I20260808 16:56:12.774709 9755 log.cc:824] T 00000000000000000000000000000000 P a7410dc5dbaf405281198230fc371f96: Log is configured to *not* fsync() on all Append() calls
I20260808 16:56:12.775159 9758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:12.773291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.181:43339 (local address 127.0.0.1:43647)
0808 16:56:12.773460 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:12.773469 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:12.773499 (+ 30us) server_negotiation.cc:407] Connection header received
0808 16:56:12.773999 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:12.774009 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:12.774045 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:12.774194 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:12.774712 (+ 518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:12.774715 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:12.774716 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 16:56:12.774720 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:12.774765 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 16:56:12.774779 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:12.774858 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:12.774899 (+ 41us) server_negotiation.cc:299] Negotiation successful
0808 16:56:12.775019 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":72,"threads_started":1}
I20260808 16:56:12.775400 9755 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7410dc5dbaf405281198230fc371f96: No bootstrap required, opened a new log
I20260808 16:56:12.776051 8288 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 16:56:12.777379 9755 raft_consensus.cc:348] T 00000000000000000000000000000000 P a7410dc5dbaf405281198230fc371f96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7410dc5dbaf405281198230fc371f96" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43647 } }
I20260808 16:56:12.777514 9755 raft_consensus.cc:374] T 00000000000000000000000000000000 P a7410dc5dbaf405281198230fc371f96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:56:12.777547 9755 raft_consensus.cc:729] T 00000000000000000000000000000000 P a7410dc5dbaf405281198230fc371f96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7410dc5dbaf405281198230fc371f96, State: Initialized, Role: FOLLOWER
I20260808 16:56:12.777655 9755 consensus_queue.cc:260] T 00000000000000000000000000000000 P a7410dc5dbaf405281198230fc371f96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7410dc5dbaf405281198230fc371f96" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43647 } }
I20260808 16:56:12.777734 9755 raft_consensus.cc:388] T 00000000000000000000000000000000 P a7410dc5dbaf405281198230fc371f96 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 16:56:12.777766 9755 raft_consensus.cc:482] T 00000000000000000000000000000000 P a7410dc5dbaf405281198230fc371f96 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 16:56:12.777813 9755 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a7410dc5dbaf405281198230fc371f96 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:56:12.778407 9758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:12.777242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.181:58907 (local address 127.0.0.1:43647)
0808 16:56:12.777274 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:12.777275 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:12.777462 (+ 187us) server_negotiation.cc:407] Connection header received
0808 16:56:12.777519 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:12.777520 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:12.777523 (+ 3us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:12.777567 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:12.778152 (+ 585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:12.778153 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:12.778154 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 16:56:12.778157 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:12.778168 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 16:56:12.778172 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:12.778226 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:12.778273 (+ 47us) server_negotiation.cc:299] Negotiation successful
0808 16:56:12.778285 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 16:56:12.778439 9755 raft_consensus.cc:504] T 00000000000000000000000000000000 P a7410dc5dbaf405281198230fc371f96 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7410dc5dbaf405281198230fc371f96" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43647 } }
I20260808 16:56:12.778538 9755 leader_election.cc:302] T 00000000000000000000000000000000 P a7410dc5dbaf405281198230fc371f96 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a7410dc5dbaf405281198230fc371f96; no voters:
I20260808 16:56:12.778692 9755 leader_election.cc:288] T 00000000000000000000000000000000 P a7410dc5dbaf405281198230fc371f96 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 16:56:12.778810 9767 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a7410dc5dbaf405281198230fc371f96 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:56:12.778928 9767 raft_consensus.cc:686] T 00000000000000000000000000000000 P a7410dc5dbaf405281198230fc371f96 [term 1 LEADER]: Becoming Leader. State: Replica: a7410dc5dbaf405281198230fc371f96, State: Running, Role: LEADER
I20260808 16:56:12.778952 9755 sys_catalog.cc:564] T 00000000000000000000000000000000 P a7410dc5dbaf405281198230fc371f96 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 16:56:12.779074 9767 consensus_queue.cc:237] T 00000000000000000000000000000000 P a7410dc5dbaf405281198230fc371f96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7410dc5dbaf405281198230fc371f96" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43647 } }
I20260808 16:56:12.779429 9768 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7410dc5dbaf405281198230fc371f96 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a7410dc5dbaf405281198230fc371f96" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7410dc5dbaf405281198230fc371f96" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43647 } } }
I20260808 16:56:12.779539 9768 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7410dc5dbaf405281198230fc371f96 [sys.catalog]: This master's current role is: LEADER
I20260808 16:56:12.779726 9770 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7410dc5dbaf405281198230fc371f96 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a7410dc5dbaf405281198230fc371f96. Latest consensus state: current_term: 1 leader_uuid: "a7410dc5dbaf405281198230fc371f96" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7410dc5dbaf405281198230fc371f96" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43647 } } }
I20260808 16:56:12.779820 9770 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7410dc5dbaf405281198230fc371f96 [sys.catalog]: This master's current role is: LEADER
I20260808 16:56:12.780236 9784 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 16:56:12.780725 9784 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 16:56:12.782056 9784 catalog_manager.cc:1347] Generated new cluster ID: bdc7978f08fa4ddcb86cdf3ef4e2238c
I20260808 16:56:12.782111 9784 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 16:56:12.799201 9784 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 16:56:12.799340 9784 catalog_manager.cc:1504] Loading token signing keys...
I20260808 16:56:12.803843 9784 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a7410dc5dbaf405281198230fc371f96: Generated new TSK 0
I20260808 16:56:12.803989 9784 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 16:56:12.813272 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 9661
2026-08-08T16:56:12Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1382 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786207919384338-8288-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 16:56:12 dist-test-slave-6zbq krb5kdc[9804](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 16:56:12 dist-test-slave-6zbq krb5kdc[9804](info): set up 2 sockets
Aug 08 16:56:12 dist-test-slave-6zbq krb5kdc[9804](info): commencing operation
krb5kdc: starting...
W20260808 16:56:13.938541 8288 mini_kdc.cc:121] Time spent starting KDC: real 1.101s user 0.005s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786207919384338-8288-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786207919384338-8288-0/krb5kdc/test-user.keytab.
Aug 08 16:56:14 dist-test-slave-6zbq krb5kdc[9804](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208174, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T16:56:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T16:56:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.24.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
I20260808 16:56:14.042241 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786207919384338-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786207919384338-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786207919384338-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:34453
--webserver_interface=127.8.24.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:37455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:34453
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:56:14.120203 9825 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:56:14.120378 9825 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:56:14.120409 9825 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:56:14.121505 9825 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 16:56:14.121543 9825 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:56:14.121562 9825 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:56:14.121578 9825 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 16:56:14.121595 9825 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 16:56:14.123059 9825 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:37455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786207919384338-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786207919384338-8288-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:34453
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:34453
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.24.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9825
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786207919384338-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:56:14.123283 9825 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:56:14.123476 9825 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:56:14.127799 9832 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:14.127835 9835 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:14.127810 9831 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:56:14.127966 9825 server_base.cc:1034] running on GCE node
I20260808 16:56:14.128162 9825 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:56:14.128401 9825 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:56:14.129531 9825 hybrid_clock.cc:648] HybridClock initialized: now 1786208174129518 us; error 28 us; skew 500 ppm
Aug 08 16:56:14 dist-test-slave-6zbq krb5kdc[9804](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208174, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:56:14.131758 9825 init.cc:377] Logged in from keytab as kudu/127.8.24.62@KRBTEST.COM (short username kudu)
I20260808 16:56:14.132143 9825 webserver.cc:460] Webserver started at http://127.8.24.62:45007/ using document root <none> and password file <none>
I20260808 16:56:14.132344 9825 fs_manager.cc:359] Metadata directory not provided
I20260808 16:56:14.132388 9825 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:56:14.132507 9825 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:56:14.133430 9825 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786207919384338-8288-0/minicluster-data/master-0/data/instance:
uuid: "dbb4b61030e1464ca3b77b5ab06b1b5a"
format_stamp: "Formatted at 2026-08-08 16:56:14 on dist-test-slave-6zbq"
server_key: "cf6661b256653875f113eec9fff7156d"
server_key_iv: "29fbb8cb2004c27fb69621d4de7e5894"
server_key_version: "encryptionkey@0"
I20260808 16:56:14.133730 9825 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786207919384338-8288-0/minicluster-data/master-0/wal/instance:
uuid: "dbb4b61030e1464ca3b77b5ab06b1b5a"
format_stamp: "Formatted at 2026-08-08 16:56:14 on dist-test-slave-6zbq"
server_key: "cf6661b256653875f113eec9fff7156d"
server_key_iv: "29fbb8cb2004c27fb69621d4de7e5894"
server_key_version: "encryptionkey@0"
I20260808 16:56:14.134923 9825 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 16:56:14.135541 9849 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:14.135679 9825 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 16:56:14.135763 9825 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786207919384338-8288-0/minicluster-data/master-0/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786207919384338-8288-0/minicluster-data/master-0/wal
uuid: "dbb4b61030e1464ca3b77b5ab06b1b5a"
format_stamp: "Formatted at 2026-08-08 16:56:14 on dist-test-slave-6zbq"
server_key: "cf6661b256653875f113eec9fff7156d"
server_key_iv: "29fbb8cb2004c27fb69621d4de7e5894"
server_key_version: "encryptionkey@0"
I20260808 16:56:14.135818 9825 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786207919384338-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786207919384338-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786207919384338-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:56:14.151000 9825 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:56:14.152825 9825 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:56:14.152943 9825 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:56:14.156462 9825 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.62:34453
I20260808 16:56:14.156503 9923 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.62:34453 every 8 connection(s)
I20260808 16:56:14.156785 9825 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
I20260808 16:56:14.157542 9924 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:56:14.159543 9924 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dbb4b61030e1464ca3b77b5ab06b1b5a: Bootstrap starting.
I20260808 16:56:14.160053 9924 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dbb4b61030e1464ca3b77b5ab06b1b5a: Neither blocks nor log segments found. Creating new log.
I20260808 16:56:14.160291 9924 log.cc:824] T 00000000000000000000000000000000 P dbb4b61030e1464ca3b77b5ab06b1b5a: Log is configured to *not* fsync() on all Append() calls
I20260808 16:56:14.160877 9924 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dbb4b61030e1464ca3b77b5ab06b1b5a: No bootstrap required, opened a new log
I20260808 16:56:14.162370 9924 raft_consensus.cc:348] T 00000000000000000000000000000000 P dbb4b61030e1464ca3b77b5ab06b1b5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbb4b61030e1464ca3b77b5ab06b1b5a" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 34453 } }
I20260808 16:56:14.162489 9924 raft_consensus.cc:374] T 00000000000000000000000000000000 P dbb4b61030e1464ca3b77b5ab06b1b5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:56:14.162510 9924 raft_consensus.cc:729] T 00000000000000000000000000000000 P dbb4b61030e1464ca3b77b5ab06b1b5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbb4b61030e1464ca3b77b5ab06b1b5a, State: Initialized, Role: FOLLOWER
I20260808 16:56:14.162632 9924 consensus_queue.cc:260] T 00000000000000000000000000000000 P dbb4b61030e1464ca3b77b5ab06b1b5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbb4b61030e1464ca3b77b5ab06b1b5a" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 34453 } }
I20260808 16:56:14.162695 9924 raft_consensus.cc:388] T 00000000000000000000000000000000 P dbb4b61030e1464ca3b77b5ab06b1b5a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 16:56:14.162726 9924 raft_consensus.cc:482] T 00000000000000000000000000000000 P dbb4b61030e1464ca3b77b5ab06b1b5a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 16:56:14.162770 9924 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dbb4b61030e1464ca3b77b5ab06b1b5a [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:56:14.163506 9924 raft_consensus.cc:504] T 00000000000000000000000000000000 P dbb4b61030e1464ca3b77b5ab06b1b5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbb4b61030e1464ca3b77b5ab06b1b5a" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 34453 } }
I20260808 16:56:14.163594 9924 leader_election.cc:302] T 00000000000000000000000000000000 P dbb4b61030e1464ca3b77b5ab06b1b5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dbb4b61030e1464ca3b77b5ab06b1b5a; no voters:
I20260808 16:56:14.163745 9924 leader_election.cc:288] T 00000000000000000000000000000000 P dbb4b61030e1464ca3b77b5ab06b1b5a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 16:56:14.163779 9928 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dbb4b61030e1464ca3b77b5ab06b1b5a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:56:14.163913 9928 raft_consensus.cc:686] T 00000000000000000000000000000000 P dbb4b61030e1464ca3b77b5ab06b1b5a [term 1 LEADER]: Becoming Leader. State: Replica: dbb4b61030e1464ca3b77b5ab06b1b5a, State: Running, Role: LEADER
I20260808 16:56:14.163964 9924 sys_catalog.cc:564] T 00000000000000000000000000000000 P dbb4b61030e1464ca3b77b5ab06b1b5a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 16:56:14.163998 9928 consensus_queue.cc:237] T 00000000000000000000000000000000 P dbb4b61030e1464ca3b77b5ab06b1b5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbb4b61030e1464ca3b77b5ab06b1b5a" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 34453 } }
I20260808 16:56:14.164347 9930 sys_catalog.cc:455] T 00000000000000000000000000000000 P dbb4b61030e1464ca3b77b5ab06b1b5a [sys.catalog]: SysCatalogTable state changed. Reason: New leader dbb4b61030e1464ca3b77b5ab06b1b5a. Latest consensus state: current_term: 1 leader_uuid: "dbb4b61030e1464ca3b77b5ab06b1b5a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbb4b61030e1464ca3b77b5ab06b1b5a" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 34453 } } }
I20260808 16:56:14.164330 9929 sys_catalog.cc:455] T 00000000000000000000000000000000 P dbb4b61030e1464ca3b77b5ab06b1b5a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dbb4b61030e1464ca3b77b5ab06b1b5a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbb4b61030e1464ca3b77b5ab06b1b5a" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 34453 } } }
I20260808 16:56:14.164443 9930 sys_catalog.cc:458] T 00000000000000000000000000000000 P dbb4b61030e1464ca3b77b5ab06b1b5a [sys.catalog]: This master's current role is: LEADER
I20260808 16:56:14.164449 9929 sys_catalog.cc:458] T 00000000000000000000000000000000 P dbb4b61030e1464ca3b77b5ab06b1b5a [sys.catalog]: This master's current role is: LEADER
I20260808 16:56:14.164911 9940 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 16:56:14.165371 9940 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 16:56:14.166196 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 9825
I20260808 16:56:14.166307 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786207919384338-8288-0/minicluster-data/master-0/wal/instance
I20260808 16:56:14.166488 8288 external_mini_cluster.cc:1428] Setting key e54c4b987c4f125fdb39c4e3d5dd3f47
I20260808 16:56:14.166730 9940 catalog_manager.cc:1347] Generated new cluster ID: 537893832e214545a16aaabaace35d89
I20260808 16:56:14.166780 9940 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 16:56:14 dist-test-slave-6zbq krb5kdc[9804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208174, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260808 16:56:14.174453 9940 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 16:56:14.175076 9940 catalog_manager.cc:1504] Loading token signing keys...
I20260808 16:56:14.175313 9958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:14.166899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35977 (local address 127.8.24.62:34453)
0808 16:56:14.167262 (+ 363us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:14.167271 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:14.167302 (+ 31us) server_negotiation.cc:407] Connection header received
0808 16:56:14.167392 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:14.167397 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:14.167435 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:14.167540 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:14.168132 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:14.169002 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:14.169715 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:14.170120 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:14.170217 (+ 97us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:14.172274 (+ 2057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:14.172278 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:14.172278 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:56:14.172289 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:14.173665 (+ 1376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:14.174018 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:14.174021 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:14.174022 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:14.174125 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:14.174382 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:14.174383 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:14.174383 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:14.174633 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:14.174788 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:14.174909 (+ 121us) server_negotiation.cc:299] Negotiation successful
0808 16:56:14.175125 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":32,"server-negotiator.queue_time_us":312,"thread_start_us":63,"threads_started":1}
I20260808 16:56:14.176049 8288 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 16:56:14.178154 9940 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dbb4b61030e1464ca3b77b5ab06b1b5a: Generated new TSK 0
I20260808 16:56:14.178306 9940 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 16:56:14.183182 9958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:14.177256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46167 (local address 127.8.24.62:34453)
0808 16:56:14.177292 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:14.177294 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:14.177395 (+ 101us) server_negotiation.cc:407] Connection header received
0808 16:56:14.177445 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:14.177445 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:14.177453 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:14.177496 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:14.177973 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:14.178750 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:14.179403 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:14.179718 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:14.179787 (+ 69us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:14.180734 (+ 947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:14.180735 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:14.180735 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:56:14.180741 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:14.182059 (+ 1318us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:14.182410 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:14.182411 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:14.182411 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:14.182466 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:14.182648 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:14.182649 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:14.182649 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:14.182770 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:14.182863 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:14.183031 (+ 168us) server_negotiation.cc:299] Negotiation successful
0808 16:56:14.183091 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 16:56:14.184757 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 9825
W20260808 16:56:14.198999 9800 connection.cc:535] client connection to 127.8.24.62:34453 recv error: Network error: failed to read from TLS socket (remote: 127.8.24.62:34453): Connection reset by peer (error 104)
2026-08-08T16:56:14Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1374 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786207919384338-8288-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 16:56:14 dist-test-slave-6zbq krb5kdc[9966](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 16:56:14 dist-test-slave-6zbq krb5kdc[9966](info): set up 2 sockets
Aug 08 16:56:14 dist-test-slave-6zbq krb5kdc[9966](info): commencing operation
krb5kdc: starting...
W20260808 16:56:15.305536 8288 mini_kdc.cc:121] Time spent starting KDC: real 1.093s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786207919384338-8288-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786207919384338-8288-0/krb5kdc/test-user.keytab.
Aug 08 16:56:15 dist-test-slave-6zbq krb5kdc[9966](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208175, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T16:56:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T16:56:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.24.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
I20260808 16:56:15.409360 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786207919384338-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786207919384338-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786207919384338-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:44867
--webserver_interface=127.8.24.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:44931
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:44867
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:56:15.485751 9985 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:56:15.485918 9985 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:56:15.485939 9985 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:56:15.487026 9985 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 16:56:15.487064 9985 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:56:15.487077 9985 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:56:15.487100 9985 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 16:56:15.487123 9985 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 16:56:15.488513 9985 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:44931
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786207919384338-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786207919384338-8288-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:44867
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:44867
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.24.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9985
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786207919384338-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:56:15.488714 9985 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:56:15.488900 9985 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:56:15.493080 9997 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:15.493084 9993 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:15.493100 9994 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:56:15.493383 9985 server_base.cc:1034] running on GCE node
I20260808 16:56:15.493569 9985 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:56:15.493777 9985 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:56:15.494896 9985 hybrid_clock.cc:648] HybridClock initialized: now 1786208175494887 us; error 28 us; skew 500 ppm
Aug 08 16:56:15 dist-test-slave-6zbq krb5kdc[9966](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208175, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:56:15.497020 9985 init.cc:377] Logged in from keytab as kudu/127.8.24.62@KRBTEST.COM (short username kudu)
I20260808 16:56:15.497411 9985 webserver.cc:460] Webserver started at http://127.8.24.62:40985/ using document root <none> and password file <none>
I20260808 16:56:15.497586 9985 fs_manager.cc:359] Metadata directory not provided
I20260808 16:56:15.497630 9985 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:56:15.497750 9985 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:56:15.498708 9985 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786207919384338-8288-0/minicluster-data/master-0/data/instance:
uuid: "e7de25775e4e487f9c97f62f094d830e"
format_stamp: "Formatted at 2026-08-08 16:56:15 on dist-test-slave-6zbq"
server_key: "bf7d81b10d624e5a4b6816f375f27d31"
server_key_iv: "3cedcb7bef6b13782e6c1694687e50b9"
server_key_version: "encryptionkey@0"
I20260808 16:56:15.499011 9985 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786207919384338-8288-0/minicluster-data/master-0/wal/instance:
uuid: "e7de25775e4e487f9c97f62f094d830e"
format_stamp: "Formatted at 2026-08-08 16:56:15 on dist-test-slave-6zbq"
server_key: "bf7d81b10d624e5a4b6816f375f27d31"
server_key_iv: "3cedcb7bef6b13782e6c1694687e50b9"
server_key_version: "encryptionkey@0"
I20260808 16:56:15.500135 9985 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 16:56:15.500741 10007 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:15.500880 9985 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 16:56:15.500959 9985 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786207919384338-8288-0/minicluster-data/master-0/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786207919384338-8288-0/minicluster-data/master-0/wal
uuid: "e7de25775e4e487f9c97f62f094d830e"
format_stamp: "Formatted at 2026-08-08 16:56:15 on dist-test-slave-6zbq"
server_key: "bf7d81b10d624e5a4b6816f375f27d31"
server_key_iv: "3cedcb7bef6b13782e6c1694687e50b9"
server_key_version: "encryptionkey@0"
I20260808 16:56:15.501016 9985 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786207919384338-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786207919384338-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786207919384338-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:56:15.508730 9985 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:56:15.510159 9985 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:56:15.510280 9985 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:56:15.513935 9985 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.62:44867
I20260808 16:56:15.513975 10088 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.62:44867 every 8 connection(s)
I20260808 16:56:15.514240 9985 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
I20260808 16:56:15.515005 10089 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:56:15.516943 10089 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e7de25775e4e487f9c97f62f094d830e: Bootstrap starting.
I20260808 16:56:15.517447 10089 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e7de25775e4e487f9c97f62f094d830e: Neither blocks nor log segments found. Creating new log.
I20260808 16:56:15.517700 10089 log.cc:824] T 00000000000000000000000000000000 P e7de25775e4e487f9c97f62f094d830e: Log is configured to *not* fsync() on all Append() calls
I20260808 16:56:15.518301 10089 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e7de25775e4e487f9c97f62f094d830e: No bootstrap required, opened a new log
I20260808 16:56:15.519784 10089 raft_consensus.cc:348] T 00000000000000000000000000000000 P e7de25775e4e487f9c97f62f094d830e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7de25775e4e487f9c97f62f094d830e" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 44867 } }
I20260808 16:56:15.519907 10089 raft_consensus.cc:374] T 00000000000000000000000000000000 P e7de25775e4e487f9c97f62f094d830e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:56:15.519939 10089 raft_consensus.cc:729] T 00000000000000000000000000000000 P e7de25775e4e487f9c97f62f094d830e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7de25775e4e487f9c97f62f094d830e, State: Initialized, Role: FOLLOWER
I20260808 16:56:15.520066 10089 consensus_queue.cc:260] T 00000000000000000000000000000000 P e7de25775e4e487f9c97f62f094d830e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7de25775e4e487f9c97f62f094d830e" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 44867 } }
I20260808 16:56:15.520126 10089 raft_consensus.cc:388] T 00000000000000000000000000000000 P e7de25775e4e487f9c97f62f094d830e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 16:56:15.520161 10089 raft_consensus.cc:482] T 00000000000000000000000000000000 P e7de25775e4e487f9c97f62f094d830e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 16:56:15.520205 10089 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e7de25775e4e487f9c97f62f094d830e [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:56:15.520951 10089 raft_consensus.cc:504] T 00000000000000000000000000000000 P e7de25775e4e487f9c97f62f094d830e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7de25775e4e487f9c97f62f094d830e" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 44867 } }
I20260808 16:56:15.521040 10089 leader_election.cc:302] T 00000000000000000000000000000000 P e7de25775e4e487f9c97f62f094d830e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e7de25775e4e487f9c97f62f094d830e; no voters:
I20260808 16:56:15.521205 10089 leader_election.cc:288] T 00000000000000000000000000000000 P e7de25775e4e487f9c97f62f094d830e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 16:56:15.521238 10094 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e7de25775e4e487f9c97f62f094d830e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:56:15.521369 10094 raft_consensus.cc:686] T 00000000000000000000000000000000 P e7de25775e4e487f9c97f62f094d830e [term 1 LEADER]: Becoming Leader. State: Replica: e7de25775e4e487f9c97f62f094d830e, State: Running, Role: LEADER
I20260808 16:56:15.521414 10089 sys_catalog.cc:564] T 00000000000000000000000000000000 P e7de25775e4e487f9c97f62f094d830e [sys.catalog]: configured and running, proceeding with master startup.
I20260808 16:56:15.521454 10094 consensus_queue.cc:237] T 00000000000000000000000000000000 P e7de25775e4e487f9c97f62f094d830e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7de25775e4e487f9c97f62f094d830e" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 44867 } }
I20260808 16:56:15.521809 10096 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7de25775e4e487f9c97f62f094d830e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e7de25775e4e487f9c97f62f094d830e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7de25775e4e487f9c97f62f094d830e" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 44867 } } }
I20260808 16:56:15.521811 10097 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7de25775e4e487f9c97f62f094d830e [sys.catalog]: SysCatalogTable state changed. Reason: New leader e7de25775e4e487f9c97f62f094d830e. Latest consensus state: current_term: 1 leader_uuid: "e7de25775e4e487f9c97f62f094d830e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7de25775e4e487f9c97f62f094d830e" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 44867 } } }
I20260808 16:56:15.521910 10097 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7de25775e4e487f9c97f62f094d830e [sys.catalog]: This master's current role is: LEADER
I20260808 16:56:15.522078 10096 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7de25775e4e487f9c97f62f094d830e [sys.catalog]: This master's current role is: LEADER
I20260808 16:56:15.522384 10107 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 16:56:15.522835 10107 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 16:56:15.522969 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 9985
I20260808 16:56:15.523082 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786207919384338-8288-0/minicluster-data/master-0/wal/instance
I20260808 16:56:15.523270 8288 external_mini_cluster.cc:1428] Setting key 9557ab9b2748647061423cd95fd8571b
I20260808 16:56:15.524313 10107 catalog_manager.cc:1347] Generated new cluster ID: 3a0c92fc8b2a48b5a65edbc70af981e0
I20260808 16:56:15.524360 10107 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 16:56:15.527912 10120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:15.523775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45985 (local address 127.8.24.62:44867)
0808 16:56:15.524027 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:15.524034 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:15.524056 (+ 22us) server_negotiation.cc:407] Connection header received
0808 16:56:15.524147 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:15.524151 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:15.524190 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:15.524347 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:15.524536 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:15.525403 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:15.526150 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:15.526651 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:15.526746 (+ 95us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:15.527330 (+ 584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:15.527334 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:15.527335 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 16:56:15.527489 (+ 154us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:15.527515 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 16:56:15.527526 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:15.527604 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:15.527692 (+ 88us) server_negotiation.cc:299] Negotiation successful
0808 16:56:15.527717 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":218,"thread_start_us":78,"threads_started":1}
I20260808 16:56:15.529558 8288 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 16:56:15.536938 10120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:15.530776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56891 (local address 127.8.24.62:44867)
0808 16:56:15.530815 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:15.530817 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:15.534534 (+ 3717us) server_negotiation.cc:407] Connection header received
0808 16:56:15.534588 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:15.534589 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:15.534608 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:15.534649 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:15.534789 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:15.535261 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:15.535917 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:15.536139 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:15.536191 (+ 52us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:15.536725 (+ 534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:15.536726 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:15.536727 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 16:56:15.536731 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:15.536744 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 16:56:15.536748 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:15.536797 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:15.536860 (+ 63us) server_negotiation.cc:299] Negotiation successful
0808 16:56:15.536870 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 16:56:15.541175 10107 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 16:56:15.541642 10107 catalog_manager.cc:1504] Loading token signing keys...
I20260808 16:56:15.546897 10107 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e7de25775e4e487f9c97f62f094d830e: Generated new TSK 0
I20260808 16:56:15.547044 10107 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 16:56:15.552013 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 9985
W20260808 16:56:15.565963 9964 connection.cc:535] client connection to 127.8.24.62:44867 recv error: Network error: failed to read from TLS socket (remote: 127.8.24.62:44867): Connection reset by peer (error 104)
2026-08-08T16:56:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1366 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786207919384338-8288-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 16:56:15 dist-test-slave-6zbq krb5kdc[10134](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 16:56:15 dist-test-slave-6zbq krb5kdc[10134](info): set up 2 sockets
Aug 08 16:56:15 dist-test-slave-6zbq krb5kdc[10134](info): commencing operation
krb5kdc: starting...
W20260808 16:56:16.673733 8288 mini_kdc.cc:121] Time spent starting KDC: real 1.095s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786207919384338-8288-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786207919384338-8288-0/krb5kdc/test-user.keytab.
Aug 08 16:56:16 dist-test-slave-6zbq krb5kdc[10134](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208176, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T16:56:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T16:56:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.24.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
I20260808 16:56:16.777864 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786207919384338-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786207919384338-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786207919384338-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:38927
--webserver_interface=127.8.24.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:36713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:38927
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:56:16.855172 10158 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:56:16.855345 10158 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:56:16.855366 10158 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:56:16.856446 10158 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 16:56:16.856485 10158 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:56:16.856498 10158 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:56:16.856518 10158 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 16:56:16.856540 10158 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 16:56:16.857935 10158 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:36713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786207919384338-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786207919384338-8288-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:38927
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:38927
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.24.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10158
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786207919384338-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:56:16.858139 10158 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:56:16.858325 10158 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:56:16.862658 10169 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:16.862673 10165 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:56:16.862706 10158 server_base.cc:1034] running on GCE node
W20260808 16:56:16.862689 10166 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:56:16.863119 10158 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:56:16.863364 10158 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:56:16.864499 10158 hybrid_clock.cc:648] HybridClock initialized: now 1786208176864486 us; error 25 us; skew 500 ppm
Aug 08 16:56:16 dist-test-slave-6zbq krb5kdc[10134](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208176, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:56:16.870939 10158 init.cc:377] Logged in from keytab as kudu/127.8.24.62@KRBTEST.COM (short username kudu)
I20260808 16:56:16.871335 10158 webserver.cc:460] Webserver started at http://127.8.24.62:37881/ using document root <none> and password file <none>
I20260808 16:56:16.871511 10158 fs_manager.cc:359] Metadata directory not provided
I20260808 16:56:16.871555 10158 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:56:16.871672 10158 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:56:16.872581 10158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786207919384338-8288-0/minicluster-data/master-0/data/instance:
uuid: "26e9161d66834732b75f9cebd340bc1c"
format_stamp: "Formatted at 2026-08-08 16:56:16 on dist-test-slave-6zbq"
server_key: "53c0c08f7ec5943ef8f396caa33b63a3"
server_key_iv: "854c5f17bf8d772d94fae3b8fa6c3fc1"
server_key_version: "encryptionkey@0"
I20260808 16:56:16.872891 10158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786207919384338-8288-0/minicluster-data/master-0/wal/instance:
uuid: "26e9161d66834732b75f9cebd340bc1c"
format_stamp: "Formatted at 2026-08-08 16:56:16 on dist-test-slave-6zbq"
server_key: "53c0c08f7ec5943ef8f396caa33b63a3"
server_key_iv: "854c5f17bf8d772d94fae3b8fa6c3fc1"
server_key_version: "encryptionkey@0"
I20260808 16:56:16.874014 10158 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 16:56:16.874665 10178 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:16.874811 10158 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:16.874890 10158 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786207919384338-8288-0/minicluster-data/master-0/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786207919384338-8288-0/minicluster-data/master-0/wal
uuid: "26e9161d66834732b75f9cebd340bc1c"
format_stamp: "Formatted at 2026-08-08 16:56:16 on dist-test-slave-6zbq"
server_key: "53c0c08f7ec5943ef8f396caa33b63a3"
server_key_iv: "854c5f17bf8d772d94fae3b8fa6c3fc1"
server_key_version: "encryptionkey@0"
I20260808 16:56:16.874948 10158 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786207919384338-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786207919384338-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786207919384338-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:56:16.877902 10158 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:56:16.879482 10158 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:56:16.879597 10158 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:56:16.883258 10158 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.62:38927
I20260808 16:56:16.883287 10252 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.62:38927 every 8 connection(s)
I20260808 16:56:16.883553 10158 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
I20260808 16:56:16.884397 10253 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:56:16.886396 10253 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26e9161d66834732b75f9cebd340bc1c: Bootstrap starting.
I20260808 16:56:16.886896 10253 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 26e9161d66834732b75f9cebd340bc1c: Neither blocks nor log segments found. Creating new log.
I20260808 16:56:16.887115 10253 log.cc:824] T 00000000000000000000000000000000 P 26e9161d66834732b75f9cebd340bc1c: Log is configured to *not* fsync() on all Append() calls
I20260808 16:56:16.887693 10253 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26e9161d66834732b75f9cebd340bc1c: No bootstrap required, opened a new log
I20260808 16:56:16.889166 10253 raft_consensus.cc:348] T 00000000000000000000000000000000 P 26e9161d66834732b75f9cebd340bc1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26e9161d66834732b75f9cebd340bc1c" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 38927 } }
I20260808 16:56:16.889289 10253 raft_consensus.cc:374] T 00000000000000000000000000000000 P 26e9161d66834732b75f9cebd340bc1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:56:16.889317 10253 raft_consensus.cc:729] T 00000000000000000000000000000000 P 26e9161d66834732b75f9cebd340bc1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26e9161d66834732b75f9cebd340bc1c, State: Initialized, Role: FOLLOWER
I20260808 16:56:16.889405 10253 consensus_queue.cc:260] T 00000000000000000000000000000000 P 26e9161d66834732b75f9cebd340bc1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26e9161d66834732b75f9cebd340bc1c" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 38927 } }
I20260808 16:56:16.889451 10253 raft_consensus.cc:388] T 00000000000000000000000000000000 P 26e9161d66834732b75f9cebd340bc1c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 16:56:16.889474 10253 raft_consensus.cc:482] T 00000000000000000000000000000000 P 26e9161d66834732b75f9cebd340bc1c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 16:56:16.889503 10253 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 26e9161d66834732b75f9cebd340bc1c [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:56:16.890211 10253 raft_consensus.cc:504] T 00000000000000000000000000000000 P 26e9161d66834732b75f9cebd340bc1c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26e9161d66834732b75f9cebd340bc1c" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 38927 } }
I20260808 16:56:16.890293 10253 leader_election.cc:302] T 00000000000000000000000000000000 P 26e9161d66834732b75f9cebd340bc1c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 26e9161d66834732b75f9cebd340bc1c; no voters:
I20260808 16:56:16.890445 10253 leader_election.cc:288] T 00000000000000000000000000000000 P 26e9161d66834732b75f9cebd340bc1c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 16:56:16.890487 10258 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 26e9161d66834732b75f9cebd340bc1c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:56:16.890630 10258 raft_consensus.cc:686] T 00000000000000000000000000000000 P 26e9161d66834732b75f9cebd340bc1c [term 1 LEADER]: Becoming Leader. State: Replica: 26e9161d66834732b75f9cebd340bc1c, State: Running, Role: LEADER
I20260808 16:56:16.890681 10253 sys_catalog.cc:564] T 00000000000000000000000000000000 P 26e9161d66834732b75f9cebd340bc1c [sys.catalog]: configured and running, proceeding with master startup.
I20260808 16:56:16.890712 10258 consensus_queue.cc:237] T 00000000000000000000000000000000 P 26e9161d66834732b75f9cebd340bc1c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26e9161d66834732b75f9cebd340bc1c" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 38927 } }
I20260808 16:56:16.891052 10259 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26e9161d66834732b75f9cebd340bc1c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "26e9161d66834732b75f9cebd340bc1c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26e9161d66834732b75f9cebd340bc1c" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 38927 } } }
I20260808 16:56:16.891058 10263 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26e9161d66834732b75f9cebd340bc1c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 26e9161d66834732b75f9cebd340bc1c. Latest consensus state: current_term: 1 leader_uuid: "26e9161d66834732b75f9cebd340bc1c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26e9161d66834732b75f9cebd340bc1c" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 38927 } } }
I20260808 16:56:16.891206 10263 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26e9161d66834732b75f9cebd340bc1c [sys.catalog]: This master's current role is: LEADER
I20260808 16:56:16.891397 10278 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 16:56:16.891557 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 10158
I20260808 16:56:16.891638 10259 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26e9161d66834732b75f9cebd340bc1c [sys.catalog]: This master's current role is: LEADER
I20260808 16:56:16.891667 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786207919384338-8288-0/minicluster-data/master-0/wal/instance
I20260808 16:56:16.891852 10278 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 16:56:16.891861 8288 external_mini_cluster.cc:1428] Setting key 79eaeaa554efbe14d2d9bce089114989
I20260808 16:56:16.893206 10278 catalog_manager.cc:1347] Generated new cluster ID: 814f1108b486419abf2e1f2b721d861d
I20260808 16:56:16.893256 10278 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 16:56:16.899785 10291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:16.892406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52523 (local address 127.8.24.62:38927)
0808 16:56:16.892561 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:16.892569 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:16.898542 (+ 5973us) server_negotiation.cc:407] Connection header received
0808 16:56:16.898668 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:16.898674 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:16.898714 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:16.898862 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:16.899395 (+ 533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:16.899398 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:16.899399 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 16:56:16.899403 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:16.899440 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 16:56:16.899454 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:16.899521 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:16.899572 (+ 51us) server_negotiation.cc:299] Negotiation successful
0808 16:56:16.899617 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":67,"threads_started":1}
I20260808 16:56:16.900657 8288 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 16:56:16.903102 10291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:16.902109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57749 (local address 127.8.24.62:38927)
0808 16:56:16.902149 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:16.902151 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:16.902301 (+ 150us) server_negotiation.cc:407] Connection header received
0808 16:56:16.902375 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:16.902375 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:16.902380 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:16.902428 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:16.902915 (+ 487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:16.902917 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:16.902917 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 16:56:16.902921 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:16.902935 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 16:56:16.902939 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:16.902992 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:16.903032 (+ 40us) server_negotiation.cc:299] Negotiation successful
0808 16:56:16.903044 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 16:56:16.905158 10278 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 16:56:16.905289 10278 catalog_manager.cc:1504] Loading token signing keys...
I20260808 16:56:16.910250 10278 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 26e9161d66834732b75f9cebd340bc1c: Generated new TSK 0
I20260808 16:56:16.910440 10278 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 16:56:16.918817 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 10158
2026-08-08T16:56:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1364 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786207919384338-8288-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 16:56:16 dist-test-slave-6zbq krb5kdc[10308](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 16:56:16 dist-test-slave-6zbq krb5kdc[10308](info): set up 2 sockets
Aug 08 16:56:16 dist-test-slave-6zbq krb5kdc[10308](info): commencing operation
krb5kdc: starting...
W20260808 16:56:18.040369 8288 mini_kdc.cc:121] Time spent starting KDC: real 1.096s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786207919384338-8288-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786207919384338-8288-0/krb5kdc/test-user.keytab.
Aug 08 16:56:18 dist-test-slave-6zbq krb5kdc[10308](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208178, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T16:56:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T16:56:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.24.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
I20260808 16:56:18.144721 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786207919384338-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786207919384338-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786207919384338-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:43415
--webserver_interface=127.8.24.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:46679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:43415
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:56:18.222061 10329 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:56:18.222232 10329 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:56:18.222254 10329 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:56:18.223343 10329 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 16:56:18.223384 10329 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:56:18.223397 10329 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:56:18.223419 10329 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 16:56:18.223441 10329 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 16:56:18.224824 10329 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:46679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786207919384338-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786207919384338-8288-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:43415
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:43415
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.24.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10329
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786207919384338-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:56:18.225030 10329 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:56:18.225220 10329 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:56:18.229521 10338 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:18.229529 10342 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:56:18.229557 10329 server_base.cc:1034] running on GCE node
W20260808 16:56:18.229529 10340 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:56:18.229972 10329 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:56:18.230230 10329 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:56:18.231359 10329 hybrid_clock.cc:648] HybridClock initialized: now 1786208178231345 us; error 33 us; skew 500 ppm
Aug 08 16:56:18 dist-test-slave-6zbq krb5kdc[10308](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208178, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:56:18.233487 10329 init.cc:377] Logged in from keytab as kudu/127.8.24.62@KRBTEST.COM (short username kudu)
I20260808 16:56:18.233842 10329 webserver.cc:460] Webserver started at http://127.8.24.62:42209/ using document root <none> and password file <none>
I20260808 16:56:18.233999 10329 fs_manager.cc:359] Metadata directory not provided
I20260808 16:56:18.234035 10329 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:56:18.234114 10329 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:56:18.235039 10329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786207919384338-8288-0/minicluster-data/master-0/data/instance:
uuid: "5706b95a78ff426da3edc85f55915655"
format_stamp: "Formatted at 2026-08-08 16:56:18 on dist-test-slave-6zbq"
server_key: "92f3fb0b24db90a113c061c13a63468c"
server_key_iv: "58f1aa8efb3668efcfc06d50e8926162"
server_key_version: "encryptionkey@0"
I20260808 16:56:18.235308 10329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786207919384338-8288-0/minicluster-data/master-0/wal/instance:
uuid: "5706b95a78ff426da3edc85f55915655"
format_stamp: "Formatted at 2026-08-08 16:56:18 on dist-test-slave-6zbq"
server_key: "92f3fb0b24db90a113c061c13a63468c"
server_key_iv: "58f1aa8efb3668efcfc06d50e8926162"
server_key_version: "encryptionkey@0"
I20260808 16:56:18.236395 10329 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 16:56:18.236971 10353 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:18.237109 10329 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 16:56:18.237188 10329 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786207919384338-8288-0/minicluster-data/master-0/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786207919384338-8288-0/minicluster-data/master-0/wal
uuid: "5706b95a78ff426da3edc85f55915655"
format_stamp: "Formatted at 2026-08-08 16:56:18 on dist-test-slave-6zbq"
server_key: "92f3fb0b24db90a113c061c13a63468c"
server_key_iv: "58f1aa8efb3668efcfc06d50e8926162"
server_key_version: "encryptionkey@0"
I20260808 16:56:18.237241 10329 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786207919384338-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786207919384338-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786207919384338-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:56:18.261313 10329 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:56:18.263160 10329 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:56:18.263260 10329 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:56:18.266772 10329 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.62:43415
I20260808 16:56:18.266803 10432 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.62:43415 every 8 connection(s)
I20260808 16:56:18.267083 10329 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
I20260808 16:56:18.267827 10434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:56:18.268472 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 10329
I20260808 16:56:18.268579 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786207919384338-8288-0/minicluster-data/master-0/wal/instance
I20260808 16:56:18.268782 8288 external_mini_cluster.cc:1428] Setting key b8d9d1210ef1ba8b39ea4beb10496ca6
I20260808 16:56:18.270617 10434 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5706b95a78ff426da3edc85f55915655: Bootstrap starting.
I20260808 16:56:18.271152 10434 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5706b95a78ff426da3edc85f55915655: Neither blocks nor log segments found. Creating new log.
I20260808 16:56:18.271418 10434 log.cc:824] T 00000000000000000000000000000000 P 5706b95a78ff426da3edc85f55915655: Log is configured to *not* fsync() on all Append() calls
I20260808 16:56:18.271988 10434 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5706b95a78ff426da3edc85f55915655: No bootstrap required, opened a new log
I20260808 16:56:18.273648 10434 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5706b95a78ff426da3edc85f55915655 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5706b95a78ff426da3edc85f55915655" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 43415 } }
I20260808 16:56:18.273770 10434 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5706b95a78ff426da3edc85f55915655 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:56:18.273801 10434 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5706b95a78ff426da3edc85f55915655 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5706b95a78ff426da3edc85f55915655, State: Initialized, Role: FOLLOWER
I20260808 16:56:18.273917 10434 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5706b95a78ff426da3edc85f55915655 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5706b95a78ff426da3edc85f55915655" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 43415 } }
I20260808 16:56:18.273996 10434 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5706b95a78ff426da3edc85f55915655 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 16:56:18.274025 10434 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5706b95a78ff426da3edc85f55915655 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 16:56:18.274073 10434 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5706b95a78ff426da3edc85f55915655 [term 0 FOLLOWER]: Advancing to term 1
Aug 08 16:56:18 dist-test-slave-6zbq krb5kdc[10308](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208178, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260808 16:56:18.274698 10434 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5706b95a78ff426da3edc85f55915655 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5706b95a78ff426da3edc85f55915655" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 43415 } }
I20260808 16:56:18.274796 10434 leader_election.cc:302] T 00000000000000000000000000000000 P 5706b95a78ff426da3edc85f55915655 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5706b95a78ff426da3edc85f55915655; no voters:
I20260808 16:56:18.274936 10434 leader_election.cc:288] T 00000000000000000000000000000000 P 5706b95a78ff426da3edc85f55915655 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 16:56:18.274976 10439 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5706b95a78ff426da3edc85f55915655 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:56:18.275120 10439 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5706b95a78ff426da3edc85f55915655 [term 1 LEADER]: Becoming Leader. State: Replica: 5706b95a78ff426da3edc85f55915655, State: Running, Role: LEADER
I20260808 16:56:18.275270 10434 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5706b95a78ff426da3edc85f55915655 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 16:56:18.275251 10439 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5706b95a78ff426da3edc85f55915655 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5706b95a78ff426da3edc85f55915655" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 43415 } }
I20260808 16:56:18.275586 10439 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5706b95a78ff426da3edc85f55915655 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5706b95a78ff426da3edc85f55915655. Latest consensus state: current_term: 1 leader_uuid: "5706b95a78ff426da3edc85f55915655" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5706b95a78ff426da3edc85f55915655" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 43415 } } }
I20260808 16:56:18.275650 10439 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5706b95a78ff426da3edc85f55915655 [sys.catalog]: This master's current role is: LEADER
I20260808 16:56:18.275864 10449 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 16:56:18.276391 10440 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5706b95a78ff426da3edc85f55915655 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5706b95a78ff426da3edc85f55915655" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5706b95a78ff426da3edc85f55915655" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 43415 } } }
I20260808 16:56:18.276501 10440 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5706b95a78ff426da3edc85f55915655 [sys.catalog]: This master's current role is: LEADER
I20260808 16:56:18.276697 10449 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 16:56:18.278187 10449 catalog_manager.cc:1347] Generated new cluster ID: 2babecc36546406386ab5fb86c28489c
I20260808 16:56:18.278234 10449 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 16:56:18.278891 10437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:18.269242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33307 (local address 127.8.24.62:43415)
0808 16:56:18.269390 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:18.269397 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:18.269420 (+ 23us) server_negotiation.cc:407] Connection header received
0808 16:56:18.269782 (+ 362us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:18.269788 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:18.269830 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:18.269927 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:18.270657 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:18.271557 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:18.272326 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:18.272698 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:18.272793 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:18.275013 (+ 2220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:18.275017 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:18.275018 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:56:18.275028 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:18.277381 (+ 2353us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:18.277690 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:18.277693 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:18.277693 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:18.277801 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:18.278013 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:18.278014 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:18.278014 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:18.278288 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:18.278465 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:18.278568 (+ 103us) server_negotiation.cc:299] Negotiation successful
0808 16:56:18.278699 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":58,"threads_started":1}
I20260808 16:56:18.283239 8288 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 16:56:18.291062 10437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:18.284559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58097 (local address 127.8.24.62:43415)
0808 16:56:18.284593 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:18.284595 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:18.284786 (+ 191us) server_negotiation.cc:407] Connection header received
0808 16:56:18.284816 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:18.284817 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:18.284826 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:18.284888 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:18.285413 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:18.286138 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:18.287074 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:18.287396 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:18.287429 (+ 33us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:18.288630 (+ 1201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:18.288632 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:18.288632 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:56:18.288639 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:18.290036 (+ 1397us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:18.290389 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:18.290390 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:18.290390 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:18.290446 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:18.290614 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:18.290614 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:18.290614 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:18.290727 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:18.290813 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:18.290909 (+ 96us) server_negotiation.cc:299] Negotiation successful
0808 16:56:18.290979 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 16:56:18.293833 10449 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 16:56:18.294507 10449 catalog_manager.cc:1504] Loading token signing keys...
I20260808 16:56:18.301492 10449 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5706b95a78ff426da3edc85f55915655: Generated new TSK 0
I20260808 16:56:18.301646 10449 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 16:56:18.307050 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 10329
2026-08-08T16:56:18Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1392 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786207919384338-8288-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 16:56:18 dist-test-slave-6zbq krb5kdc[10481](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 16:56:18 dist-test-slave-6zbq krb5kdc[10481](info): set up 2 sockets
Aug 08 16:56:18 dist-test-slave-6zbq krb5kdc[10481](info): commencing operation
krb5kdc: starting...
W20260808 16:56:19.437150 8288 mini_kdc.cc:121] Time spent starting KDC: real 1.100s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786207919384338-8288-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786207919384338-8288-0/krb5kdc/test-user.keytab.
Aug 08 16:56:19 dist-test-slave-6zbq krb5kdc[10481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208179, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T16:56:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T16:56:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.24.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
I20260808 16:56:19.542151 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786207919384338-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786207919384338-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786207919384338-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:44661
--webserver_interface=127.8.24.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:45401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:44661
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:56:19.619202 10501 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:56:19.619401 10501 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:56:19.619431 10501 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:56:19.620532 10501 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 16:56:19.620572 10501 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:56:19.620591 10501 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:56:19.620606 10501 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 16:56:19.620622 10501 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 16:56:19.622045 10501 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:45401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786207919384338-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786207919384338-8288-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:44661
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:44661
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.24.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10501
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786207919384338-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:56:19.622287 10501 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:56:19.622501 10501 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:56:19.626912 10514 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:19.626921 10509 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:56:19.626937 10501 server_base.cc:1034] running on GCE node
W20260808 16:56:19.627005 10510 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:56:19.627348 10501 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:56:19.627588 10501 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:56:19.628726 10501 hybrid_clock.cc:648] HybridClock initialized: now 1786208179628707 us; error 34 us; skew 500 ppm
Aug 08 16:56:19 dist-test-slave-6zbq krb5kdc[10481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208179, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:56:19.630949 10501 init.cc:377] Logged in from keytab as kudu/127.8.24.62@KRBTEST.COM (short username kudu)
I20260808 16:56:19.631358 10501 webserver.cc:460] Webserver started at http://127.8.24.62:35681/ using document root <none> and password file <none>
I20260808 16:56:19.631539 10501 fs_manager.cc:359] Metadata directory not provided
I20260808 16:56:19.631579 10501 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:56:19.631696 10501 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:56:19.632637 10501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786207919384338-8288-0/minicluster-data/master-0/data/instance:
uuid: "846b3f653816461288c8936d4b3b5378"
format_stamp: "Formatted at 2026-08-08 16:56:19 on dist-test-slave-6zbq"
server_key: "ccec037c016e31889a58162b1658d236"
server_key_iv: "d4abd0a14ff922b1d5ee8f5cc2c2d5f8"
server_key_version: "encryptionkey@0"
I20260808 16:56:19.632948 10501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786207919384338-8288-0/minicluster-data/master-0/wal/instance:
uuid: "846b3f653816461288c8936d4b3b5378"
format_stamp: "Formatted at 2026-08-08 16:56:19 on dist-test-slave-6zbq"
server_key: "ccec037c016e31889a58162b1658d236"
server_key_iv: "d4abd0a14ff922b1d5ee8f5cc2c2d5f8"
server_key_version: "encryptionkey@0"
I20260808 16:56:19.634101 10501 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 16:56:19.634799 10522 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:19.634940 10501 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:19.635017 10501 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786207919384338-8288-0/minicluster-data/master-0/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786207919384338-8288-0/minicluster-data/master-0/wal
uuid: "846b3f653816461288c8936d4b3b5378"
format_stamp: "Formatted at 2026-08-08 16:56:19 on dist-test-slave-6zbq"
server_key: "ccec037c016e31889a58162b1658d236"
server_key_iv: "d4abd0a14ff922b1d5ee8f5cc2c2d5f8"
server_key_version: "encryptionkey@0"
I20260808 16:56:19.635079 10501 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786207919384338-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786207919384338-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786207919384338-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:56:19.653568 10501 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:56:19.655144 10501 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:56:19.655267 10501 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:56:19.658891 10501 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.62:44661
I20260808 16:56:19.658921 10599 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.62:44661 every 8 connection(s)
I20260808 16:56:19.659204 10501 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
I20260808 16:56:19.659945 10600 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:56:19.662032 10600 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 846b3f653816461288c8936d4b3b5378: Bootstrap starting.
I20260808 16:56:19.662590 10600 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 846b3f653816461288c8936d4b3b5378: Neither blocks nor log segments found. Creating new log.
I20260808 16:56:19.662842 10600 log.cc:824] T 00000000000000000000000000000000 P 846b3f653816461288c8936d4b3b5378: Log is configured to *not* fsync() on all Append() calls
I20260808 16:56:19.663443 10600 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 846b3f653816461288c8936d4b3b5378: No bootstrap required, opened a new log
I20260808 16:56:19.664924 10600 raft_consensus.cc:348] T 00000000000000000000000000000000 P 846b3f653816461288c8936d4b3b5378 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "846b3f653816461288c8936d4b3b5378" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 44661 } }
I20260808 16:56:19.665050 10600 raft_consensus.cc:374] T 00000000000000000000000000000000 P 846b3f653816461288c8936d4b3b5378 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:56:19.665079 10600 raft_consensus.cc:729] T 00000000000000000000000000000000 P 846b3f653816461288c8936d4b3b5378 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 846b3f653816461288c8936d4b3b5378, State: Initialized, Role: FOLLOWER
I20260808 16:56:19.665205 10600 consensus_queue.cc:260] T 00000000000000000000000000000000 P 846b3f653816461288c8936d4b3b5378 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "846b3f653816461288c8936d4b3b5378" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 44661 } }
I20260808 16:56:19.665270 10600 raft_consensus.cc:388] T 00000000000000000000000000000000 P 846b3f653816461288c8936d4b3b5378 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 16:56:19.665299 10600 raft_consensus.cc:482] T 00000000000000000000000000000000 P 846b3f653816461288c8936d4b3b5378 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 16:56:19.665345 10600 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 846b3f653816461288c8936d4b3b5378 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:56:19.665794 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 10501
I20260808 16:56:19.665899 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786207919384338-8288-0/minicluster-data/master-0/wal/instance
I20260808 16:56:19.666077 8288 external_mini_cluster.cc:1428] Setting key e6c629562b441ba2b0723c013c72f81c
I20260808 16:56:19.666167 10600 raft_consensus.cc:504] T 00000000000000000000000000000000 P 846b3f653816461288c8936d4b3b5378 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "846b3f653816461288c8936d4b3b5378" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 44661 } }
I20260808 16:56:19.666275 10600 leader_election.cc:302] T 00000000000000000000000000000000 P 846b3f653816461288c8936d4b3b5378 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 846b3f653816461288c8936d4b3b5378; no voters:
I20260808 16:56:19.666440 10600 leader_election.cc:288] T 00000000000000000000000000000000 P 846b3f653816461288c8936d4b3b5378 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 16:56:19.666575 10605 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 846b3f653816461288c8936d4b3b5378 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:56:19.666676 10600 sys_catalog.cc:564] T 00000000000000000000000000000000 P 846b3f653816461288c8936d4b3b5378 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 16:56:19.666708 10605 raft_consensus.cc:686] T 00000000000000000000000000000000 P 846b3f653816461288c8936d4b3b5378 [term 1 LEADER]: Becoming Leader. State: Replica: 846b3f653816461288c8936d4b3b5378, State: Running, Role: LEADER
I20260808 16:56:19.666886 10605 consensus_queue.cc:237] T 00000000000000000000000000000000 P 846b3f653816461288c8936d4b3b5378 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "846b3f653816461288c8936d4b3b5378" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 44661 } }
I20260808 16:56:19.667207 10605 sys_catalog.cc:455] T 00000000000000000000000000000000 P 846b3f653816461288c8936d4b3b5378 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 846b3f653816461288c8936d4b3b5378. Latest consensus state: current_term: 1 leader_uuid: "846b3f653816461288c8936d4b3b5378" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "846b3f653816461288c8936d4b3b5378" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 44661 } } }
I20260808 16:56:19.667336 10605 sys_catalog.cc:458] T 00000000000000000000000000000000 P 846b3f653816461288c8936d4b3b5378 [sys.catalog]: This master's current role is: LEADER
I20260808 16:56:19.667392 10610 sys_catalog.cc:455] T 00000000000000000000000000000000 P 846b3f653816461288c8936d4b3b5378 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "846b3f653816461288c8936d4b3b5378" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "846b3f653816461288c8936d4b3b5378" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 44661 } } }
I20260808 16:56:19.667534 10610 sys_catalog.cc:458] T 00000000000000000000000000000000 P 846b3f653816461288c8936d4b3b5378 [sys.catalog]: This master's current role is: LEADER
I20260808 16:56:19.667817 10618 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 16:56:19.668267 10618 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 16:56:19.669584 10618 catalog_manager.cc:1347] Generated new cluster ID: a4660daea8df47a9ba02aba80bbae77f
I20260808 16:56:19.669631 10618 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 16:56:19.671716 10608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:19.666475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39885 (local address 127.8.24.62:44661)
0808 16:56:19.666793 (+ 318us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:19.666802 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:19.666838 (+ 36us) server_negotiation.cc:407] Connection header received
0808 16:56:19.666943 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:19.666950 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:19.667012 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:19.667148 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:19.667373 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:19.668716 (+ 1343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:19.669790 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:19.670373 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:19.670465 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:19.671062 (+ 597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:19.671065 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:19.671066 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 16:56:19.671268 (+ 202us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:19.671308 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 16:56:19.671321 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:19.671379 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:19.671468 (+ 89us) server_negotiation.cc:299] Negotiation successful
0808 16:56:19.671523 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":266,"thread_start_us":64,"threads_started":1}
I20260808 16:56:19.672613 8288 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 16:56:19.677124 10608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:19.674080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60015 (local address 127.8.24.62:44661)
0808 16:56:19.674108 (+ 28us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:19.674109 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:19.674138 (+ 29us) server_negotiation.cc:407] Connection header received
0808 16:56:19.674197 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:19.674197 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:19.674206 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:19.674246 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:19.674398 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:19.675070 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:19.676026 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:19.676315 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:19.676368 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:19.676919 (+ 551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:19.676920 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:19.676921 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 16:56:19.676925 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:19.676937 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 16:56:19.676941 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:19.676980 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:19.677040 (+ 60us) server_negotiation.cc:299] Negotiation successful
0808 16:56:19.677059 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13}
I20260808 16:56:19.680337 10618 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 16:56:19.680778 10618 catalog_manager.cc:1504] Loading token signing keys...
I20260808 16:56:19.684149 10618 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 846b3f653816461288c8936d4b3b5378: Generated new TSK 0
I20260808 16:56:19.684301 10618 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 16:56:19.691219 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 10501
2026-08-08T16:56:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1385 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786207919384338-8288-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 16:56:19 dist-test-slave-6zbq krb5kdc[10645](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 16:56:19 dist-test-slave-6zbq krb5kdc[10645](info): set up 2 sockets
Aug 08 16:56:19 dist-test-slave-6zbq krb5kdc[10645](info): commencing operation
krb5kdc: starting...
W20260808 16:56:20.819691 8288 mini_kdc.cc:121] Time spent starting KDC: real 1.097s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786207919384338-8288-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786207919384338-8288-0/krb5kdc/test-user.keytab.
Aug 08 16:56:20 dist-test-slave-6zbq krb5kdc[10645](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208180, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T16:56:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T16:56:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.24.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
I20260808 16:56:20.925045 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786207919384338-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786207919384338-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786207919384338-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:43813
--webserver_interface=127.8.24.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:41831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:43813
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:56:21.002249 10667 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:56:21.002439 10667 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:56:21.002461 10667 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:56:21.003507 10667 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 16:56:21.003539 10667 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:56:21.003552 10667 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:56:21.003562 10667 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 16:56:21.003573 10667 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 16:56:21.004940 10667 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:41831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786207919384338-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786207919384338-8288-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:43813
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:43813
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.24.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10667
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786207919384338-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:56:21.005113 10667 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:56:21.005271 10667 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:56:21.009579 10676 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:56:21.009645 10667 server_base.cc:1034] running on GCE node
W20260808 16:56:21.009769 10681 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:21.009588 10675 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:56:21.010040 10667 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:56:21.010294 10667 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:56:21.011447 10667 hybrid_clock.cc:648] HybridClock initialized: now 1786208181011430 us; error 34 us; skew 500 ppm
Aug 08 16:56:21 dist-test-slave-6zbq krb5kdc[10645](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208181, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:56:21.013702 10667 init.cc:377] Logged in from keytab as kudu/127.8.24.62@KRBTEST.COM (short username kudu)
I20260808 16:56:21.014079 10667 webserver.cc:460] Webserver started at http://127.8.24.62:40053/ using document root <none> and password file <none>
I20260808 16:56:21.014263 10667 fs_manager.cc:359] Metadata directory not provided
I20260808 16:56:21.014309 10667 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:56:21.014446 10667 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:56:21.015378 10667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786207919384338-8288-0/minicluster-data/master-0/data/instance:
uuid: "6cdae36db73d477e885f984fc2139836"
format_stamp: "Formatted at 2026-08-08 16:56:21 on dist-test-slave-6zbq"
server_key: "0f2fc5fcd264fa18e0d5762c1d846c04"
server_key_iv: "fb7ebf7dbb092123ccb720ca93b7f85f"
server_key_version: "encryptionkey@0"
I20260808 16:56:21.015692 10667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786207919384338-8288-0/minicluster-data/master-0/wal/instance:
uuid: "6cdae36db73d477e885f984fc2139836"
format_stamp: "Formatted at 2026-08-08 16:56:21 on dist-test-slave-6zbq"
server_key: "0f2fc5fcd264fa18e0d5762c1d846c04"
server_key_iv: "fb7ebf7dbb092123ccb720ca93b7f85f"
server_key_version: "encryptionkey@0"
I20260808 16:56:21.016808 10667 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 16:56:21.017374 10692 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:21.017516 10667 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 16:56:21.017594 10667 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786207919384338-8288-0/minicluster-data/master-0/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786207919384338-8288-0/minicluster-data/master-0/wal
uuid: "6cdae36db73d477e885f984fc2139836"
format_stamp: "Formatted at 2026-08-08 16:56:21 on dist-test-slave-6zbq"
server_key: "0f2fc5fcd264fa18e0d5762c1d846c04"
server_key_iv: "fb7ebf7dbb092123ccb720ca93b7f85f"
server_key_version: "encryptionkey@0"
I20260808 16:56:21.017661 10667 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786207919384338-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786207919384338-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786207919384338-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:56:21.020681 10667 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:56:21.022557 10667 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:56:21.022681 10667 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:56:21.026084 10667 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.62:43813
I20260808 16:56:21.026121 10760 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.62:43813 every 8 connection(s)
I20260808 16:56:21.026398 10667 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
I20260808 16:56:21.027271 10763 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:56:21.028807 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 10667
I20260808 16:56:21.028923 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786207919384338-8288-0/minicluster-data/master-0/wal/instance
I20260808 16:56:21.029111 8288 external_mini_cluster.cc:1428] Setting key 2505efd6f84ed032caff5c0637ae462e
I20260808 16:56:21.029430 10763 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6cdae36db73d477e885f984fc2139836: Bootstrap starting.
I20260808 16:56:21.030041 10763 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6cdae36db73d477e885f984fc2139836: Neither blocks nor log segments found. Creating new log.
I20260808 16:56:21.030380 10763 log.cc:824] T 00000000000000000000000000000000 P 6cdae36db73d477e885f984fc2139836: Log is configured to *not* fsync() on all Append() calls
I20260808 16:56:21.031044 10763 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6cdae36db73d477e885f984fc2139836: No bootstrap required, opened a new log
I20260808 16:56:21.031471 10767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:21.029562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50457 (local address 127.8.24.62:43813)
0808 16:56:21.029765 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:21.029773 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:21.029805 (+ 32us) server_negotiation.cc:407] Connection header received
0808 16:56:21.030322 (+ 517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:21.030332 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:21.030396 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:21.030539 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:21.031083 (+ 544us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:21.031086 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:21.031086 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 16:56:21.031091 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:21.031133 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 16:56:21.031148 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:21.031217 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:21.031266 (+ 49us) server_negotiation.cc:299] Negotiation successful
0808 16:56:21.031299 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":91,"threads_started":1}
I20260808 16:56:21.032317 8288 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 16:56:21.033345 10763 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6cdae36db73d477e885f984fc2139836 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cdae36db73d477e885f984fc2139836" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 43813 } }
I20260808 16:56:21.033514 10763 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6cdae36db73d477e885f984fc2139836 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:56:21.033558 10763 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6cdae36db73d477e885f984fc2139836 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cdae36db73d477e885f984fc2139836, State: Initialized, Role: FOLLOWER
I20260808 16:56:21.033672 10763 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6cdae36db73d477e885f984fc2139836 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cdae36db73d477e885f984fc2139836" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 43813 } }
I20260808 16:56:21.033762 10763 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6cdae36db73d477e885f984fc2139836 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 16:56:21.033793 10763 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6cdae36db73d477e885f984fc2139836 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 16:56:21.033855 10763 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6cdae36db73d477e885f984fc2139836 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:56:21.034365 10763 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6cdae36db73d477e885f984fc2139836 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cdae36db73d477e885f984fc2139836" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 43813 } }
I20260808 16:56:21.034471 10763 leader_election.cc:302] T 00000000000000000000000000000000 P 6cdae36db73d477e885f984fc2139836 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6cdae36db73d477e885f984fc2139836; no voters:
I20260808 16:56:21.034615 10763 leader_election.cc:288] T 00000000000000000000000000000000 P 6cdae36db73d477e885f984fc2139836 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 16:56:21.034731 10767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:21.033812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52921 (local address 127.8.24.62:43813)
0808 16:56:21.033844 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:21.033846 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:21.033952 (+ 106us) server_negotiation.cc:407] Connection header received
0808 16:56:21.034004 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:21.034005 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:21.034009 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:21.034053 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:21.034517 (+ 464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:21.034518 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:21.034518 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 16:56:21.034522 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:21.034533 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 16:56:21.034537 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:21.034585 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:21.034671 (+ 86us) server_negotiation.cc:299] Negotiation successful
0808 16:56:21.034682 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 16:56:21.034642 10778 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6cdae36db73d477e885f984fc2139836 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:56:21.034883 10778 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6cdae36db73d477e885f984fc2139836 [term 1 LEADER]: Becoming Leader. State: Replica: 6cdae36db73d477e885f984fc2139836, State: Running, Role: LEADER
I20260808 16:56:21.034912 10763 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6cdae36db73d477e885f984fc2139836 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 16:56:21.034973 10778 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6cdae36db73d477e885f984fc2139836 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cdae36db73d477e885f984fc2139836" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 43813 } }
I20260808 16:56:21.035284 10778 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6cdae36db73d477e885f984fc2139836 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6cdae36db73d477e885f984fc2139836" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cdae36db73d477e885f984fc2139836" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 43813 } } }
I20260808 16:56:21.035404 10778 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6cdae36db73d477e885f984fc2139836 [sys.catalog]: This master's current role is: LEADER
I20260808 16:56:21.035578 10778 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6cdae36db73d477e885f984fc2139836 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6cdae36db73d477e885f984fc2139836. Latest consensus state: current_term: 1 leader_uuid: "6cdae36db73d477e885f984fc2139836" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cdae36db73d477e885f984fc2139836" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 43813 } } }
I20260808 16:56:21.035660 10794 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 16:56:21.035681 10778 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6cdae36db73d477e885f984fc2139836 [sys.catalog]: This master's current role is: LEADER
I20260808 16:56:21.036140 10794 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 16:56:21.037524 10794 catalog_manager.cc:1347] Generated new cluster ID: 7dd768b9a21a42a99626f25f7d4d84bc
I20260808 16:56:21.037578 10794 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 16:56:21.063180 10794 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 16:56:21.063326 10794 catalog_manager.cc:1504] Loading token signing keys...
I20260808 16:56:21.071537 10794 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6cdae36db73d477e885f984fc2139836: Generated new TSK 0
I20260808 16:56:21.071691 10794 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 16:56:21.075368 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 10667
2026-08-08T16:56:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1379 ms)
[----------] 18 tests from AuthTokenIssuingTest (24992 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786207919384338-8288-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 16:56:21 dist-test-slave-6zbq krb5kdc[10814](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 16:56:21 dist-test-slave-6zbq krb5kdc[10814](info): set up 2 sockets
Aug 08 16:56:21 dist-test-slave-6zbq krb5kdc[10814](info): commencing operation
krb5kdc: starting...
W20260808 16:56:22.198971 8288 mini_kdc.cc:121] Time spent starting KDC: real 1.097s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786207919384338-8288-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786207919384338-8288-0/krb5kdc/test-user.keytab.
Aug 08 16:56:22 dist-test-slave-6zbq krb5kdc[10814](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208182, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T16:56:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T16:56:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.24.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
I20260808 16:56:22.308269 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786207919384338-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786207919384338-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786207919384338-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:34315
--webserver_interface=127.8.24.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:46585
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:34315,127.8.24.61:40469,127.8.24.60:39733
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:56:22.386109 10831 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 16:56:22.386303 10831 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:56:22.386329 10831 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:56:22.386375 10831 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:56:22.387461 10831 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 16:56:22.387501 10831 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:56:22.387521 10831 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:56:22.387535 10831 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 16:56:22.387553 10831 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 16:56:22.388983 10831 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:46585
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786207919384338-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786207919384338-8288-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:34315,127.8.24.61:40469,127.8.24.60:39733
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:34315
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.24.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10831
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786207919384338-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:56:22.389211 10831 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:56:22.389406 10831 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:56:22.393724 10839 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:22.393764 10842 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:22.393712 10840 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:56:22.393925 10831 server_base.cc:1034] running on GCE node
I20260808 16:56:22.394171 10831 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:56:22.394455 10831 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:56:22.395587 10831 hybrid_clock.cc:648] HybridClock initialized: now 1786208182395575 us; error 29 us; skew 500 ppm
Aug 08 16:56:22 dist-test-slave-6zbq krb5kdc[10814](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208182, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:56:22.401659 10831 init.cc:377] Logged in from keytab as kudu/127.8.24.62@KRBTEST.COM (short username kudu)
I20260808 16:56:22.402069 10831 webserver.cc:460] Webserver started at http://127.8.24.62:43571/ using document root <none> and password file <none>
I20260808 16:56:22.402267 10831 fs_manager.cc:359] Metadata directory not provided
I20260808 16:56:22.402310 10831 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:56:22.402446 10831 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:56:22.403344 10831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786207919384338-8288-0/minicluster-data/master-0/data/instance:
uuid: "67c8331b5fa346e0860f71d6d8e5940a"
format_stamp: "Formatted at 2026-08-08 16:56:22 on dist-test-slave-6zbq"
server_key: "c991242cc90b7a9eb4fbfcb17fdbc8fc"
server_key_iv: "47e6922cacccb9f99658798c4617d507"
server_key_version: "encryptionkey@0"
I20260808 16:56:22.403644 10831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786207919384338-8288-0/minicluster-data/master-0/wal/instance:
uuid: "67c8331b5fa346e0860f71d6d8e5940a"
format_stamp: "Formatted at 2026-08-08 16:56:22 on dist-test-slave-6zbq"
server_key: "c991242cc90b7a9eb4fbfcb17fdbc8fc"
server_key_iv: "47e6922cacccb9f99658798c4617d507"
server_key_version: "encryptionkey@0"
I20260808 16:56:22.404781 10831 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 16:56:22.405390 10852 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:22.405534 10831 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 16:56:22.405613 10831 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786207919384338-8288-0/minicluster-data/master-0/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786207919384338-8288-0/minicluster-data/master-0/wal
uuid: "67c8331b5fa346e0860f71d6d8e5940a"
format_stamp: "Formatted at 2026-08-08 16:56:22 on dist-test-slave-6zbq"
server_key: "c991242cc90b7a9eb4fbfcb17fdbc8fc"
server_key_iv: "47e6922cacccb9f99658798c4617d507"
server_key_version: "encryptionkey@0"
I20260808 16:56:22.405669 10831 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786207919384338-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786207919384338-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786207919384338-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:56:22.419533 10831 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:56:22.420989 10831 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:56:22.421113 10831 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:56:22.424649 10831 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.62:34315
I20260808 16:56:22.424682 10933 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.62:34315 every 8 connection(s)
I20260808 16:56:22.424962 10831 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
I20260808 16:56:22.425789 10934 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:56:22.427117 10934 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.24.62" port: 34315 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 16:56:22 dist-test-slave-6zbq krb5kdc[10814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208182, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260808 16:56:22.432688 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 10831
I20260808 16:56:22.432796 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786207919384338-8288-0/minicluster-data/master-0/wal/instance
I20260808 16:56:22.432952 8288 external_mini_cluster.cc:1428] Setting key e3bb0e06e32150b49ed1d69b55f1e2d6
I20260808 16:56:22.435264 10938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:22.427623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:34315 (local address 127.0.0.1:58206)
0808 16:56:22.427772 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:22.427809 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 16:56:22.428220 (+ 411us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:56:22.428455 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:56:22.428457 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:56:22.428687 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:56:22.428831 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:22.428836 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:22.429755 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:22.429756 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:22.430424 (+ 668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:22.430427 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:22.430862 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:22.430863 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:22.430925 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:22.431324 (+ 399us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:56:22.431332 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:56:22.432506 (+ 1174us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:56:22.434012 (+ 1506us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:56:22.434018 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:56:22.434020 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:56:22.434188 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:56:22.434407 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:56:22.434408 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:56:22.434408 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:56:22.434489 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:56:22.434784 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:56:22.434788 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:56:22.434846 (+ 58us) client_negotiation.cc:769] Sending connection context
0808 16:56:22.434922 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 16:56:22.435136 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"mutex_wait_us":90,"thread_start_us":69,"threads_started":1}
I20260808 16:56:22.435276 10939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:22.427672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58206 (local address 127.8.24.62:34315)
0808 16:56:22.427800 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:22.427808 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:22.427889 (+ 81us) server_negotiation.cc:407] Connection header received
0808 16:56:22.428356 (+ 467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:22.428358 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:22.428374 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:22.428446 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:22.428915 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:22.429667 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:22.430491 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:22.430777 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:22.430876 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:22.432583 (+ 1707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:22.432590 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:22.432590 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:56:22.432597 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:22.433939 (+ 1342us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:22.434262 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:22.434264 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:22.434265 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:22.434357 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:22.434541 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:22.434541 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:22.434542 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:22.434724 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:22.434793 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:22.434954 (+ 161us) server_negotiation.cc:299] Negotiation successful
0808 16:56:22.435061 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":66,"threads_started":1}
I20260808 16:56:22.436162 10934 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.24.61" port: 40469 } has no permanent_uuid. Determining permanent_uuid...
I20260808 16:56:22.436426 10938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:22.436335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.61:40469 (local address 127.0.0.1:57978)
0808 16:56:22.436364 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:22.436404 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.61:40469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
W20260808 16:56:22.436542 10853 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.24.61:40469: connect: Connection refused (error 111)
I20260808 16:56:22.436905 10938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:22.436820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.61:40469 (local address 127.0.0.1:57984)
0808 16:56:22.436861 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:22.436884 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.61:40469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260808 16:56:22.436995 10934 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.24.61:40469: Network error: Client connection negotiation failed: client connection to 127.8.24.61:40469: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.8.24.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.61.keytab.
Entry for principal kudu/127.8.24.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.61.keytab.
I20260808 16:56:22.454056 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786207919384338-8288-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786207919384338-8288-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786207919384338-8288-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786207919384338-8288-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.61:40469
--webserver_interface=127.8.24.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.61.keytab
--principal=kudu/127.8.24.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:46585
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:34315,127.8.24.61:40469,127.8.24.60:39733
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 16:56:22.486171 10934 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.24.61" port: 40469 } attempt: 1
I20260808 16:56:22.486539 10938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:22.486444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.61:40469 (local address 127.0.0.1:57994)
0808 16:56:22.486488 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:22.486518 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.61:40469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260808 16:56:22.486848 10938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:22.486744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.61:40469 (local address 127.0.0.1:58008)
0808 16:56:22.486789 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:22.486820 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.61:40469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260808 16:56:22.486949 10934 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.24.61:40469: Network error: Client connection negotiation failed: client connection to 127.8.24.61:40469: connect: Connection refused (error 111)
W20260808 16:56:22.531553 10944 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 16:56:22.531728 10944 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:56:22.531742 10944 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:56:22.531767 10944 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:56:22.532866 10944 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 16:56:22.532902 10944 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:56:22.532915 10944 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:56:22.532939 10944 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 16:56:22.532960 10944 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 16:56:22.534663 10944 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:46585
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786207919384338-8288-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786207919384338-8288-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:34315,127.8.24.61:40469,127.8.24.60:39733
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.61.keytab
--principal=kudu/127.8.24.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.61:40469
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786207919384338-8288-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.8.24.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10944
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786207919384338-8288-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:56:22.534880 10944 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:56:22.535072 10944 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:56:22.539286 10952 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:22.539309 10951 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:56:22.539333 10944 server_base.cc:1034] running on GCE node
W20260808 16:56:22.539306 10957 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:56:22.539636 10944 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:56:22.539892 10944 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:56:22.541023 10944 hybrid_clock.cc:648] HybridClock initialized: now 1786208182541010 us; error 26 us; skew 500 ppm
Aug 08 16:56:22 dist-test-slave-6zbq krb5kdc[10814](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208182, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:56:22.543188 10944 init.cc:377] Logged in from keytab as kudu/127.8.24.61@KRBTEST.COM (short username kudu)
I20260808 16:56:22.543597 10944 webserver.cc:460] Webserver started at http://127.8.24.61:34475/ using document root <none> and password file <none>
I20260808 16:56:22.543805 10944 fs_manager.cc:359] Metadata directory not provided
I20260808 16:56:22.543851 10944 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:56:22.543967 10944 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:56:22.544906 10944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786207919384338-8288-0/minicluster-data/master-1/data/instance:
uuid: "b90bb81c60a943daa7e1639eb2785c8c"
format_stamp: "Formatted at 2026-08-08 16:56:22 on dist-test-slave-6zbq"
server_key: "e446741551d4d5cdfd27a90792606ee0"
server_key_iv: "83a775ab9b98cccaa59f6ef68eb7a5f6"
server_key_version: "encryptionkey@0"
I20260808 16:56:22.545207 10944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786207919384338-8288-0/minicluster-data/master-1/wal/instance:
uuid: "b90bb81c60a943daa7e1639eb2785c8c"
format_stamp: "Formatted at 2026-08-08 16:56:22 on dist-test-slave-6zbq"
server_key: "e446741551d4d5cdfd27a90792606ee0"
server_key_iv: "83a775ab9b98cccaa59f6ef68eb7a5f6"
server_key_version: "encryptionkey@0"
I20260808 16:56:22.546387 10944 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 16:56:22.547014 10967 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:22.547158 10944 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 16:56:22.547235 10944 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786207919384338-8288-0/minicluster-data/master-1/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786207919384338-8288-0/minicluster-data/master-1/wal
uuid: "b90bb81c60a943daa7e1639eb2785c8c"
format_stamp: "Formatted at 2026-08-08 16:56:22 on dist-test-slave-6zbq"
server_key: "e446741551d4d5cdfd27a90792606ee0"
server_key_iv: "83a775ab9b98cccaa59f6ef68eb7a5f6"
server_key_version: "encryptionkey@0"
I20260808 16:56:22.547300 10944 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786207919384338-8288-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786207919384338-8288-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786207919384338-8288-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:56:22.555075 10934 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.24.61" port: 40469 } attempt: 2
I20260808 16:56:22.555433 10938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:22.555305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.61:40469 (local address 127.0.0.1:58024)
0808 16:56:22.555365 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:22.555400 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.61:40469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260808 16:56:22.555694 10938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:22.555629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.61:40469 (local address 127.0.0.1:58034)
0808 16:56:22.555660 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:22.555680 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.61:40469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260808 16:56:22.555794 10934 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.24.61:40469: Network error: Client connection negotiation failed: client connection to 127.8.24.61:40469: connect: Connection refused (error 111)
I20260808 16:56:22.566882 10944 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:56:22.568353 10944 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:56:22.568465 10944 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:56:22.572041 10944 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.61:40469
I20260808 16:56:22.572073 11038 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.61:40469 every 8 connection(s)
I20260808 16:56:22.572346 10944 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786207919384338-8288-0/minicluster-data/master-1/data/info.pb
I20260808 16:56:22.573227 11039 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:56:22.574611 11039 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.24.62" port: 34315 } has no permanent_uuid. Determining permanent_uuid...
I20260808 16:56:22.578023 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 10944
I20260808 16:56:22.578114 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786207919384338-8288-0/minicluster-data/master-1/wal/instance
I20260808 16:56:22.578305 8288 external_mini_cluster.cc:1428] Setting key ce6c5e3f7bfeffe7d70d832db84a44ca
Aug 08 16:56:22 dist-test-slave-6zbq krb5kdc[10814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208182, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.61@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260808 16:56:22.582026 10939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:22.575171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58222 (local address 127.8.24.62:34315)
0808 16:56:22.575222 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:22.575224 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:22.575376 (+ 152us) server_negotiation.cc:407] Connection header received
0808 16:56:22.575793 (+ 417us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:22.575794 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:22.575803 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:22.575850 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:22.576342 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:22.576788 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:22.577633 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:22.577810 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:22.577850 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:22.579696 (+ 1846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:22.579698 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:22.579698 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:56:22.579705 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:22.580811 (+ 1106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:22.581158 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:22.581159 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:22.581159 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:22.581214 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:22.581450 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:22.581451 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:22.581451 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:22.581561 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:22.581617 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:22.581862 (+ 245us) server_negotiation.cc:299] Negotiation successful
0808 16:56:22.581943 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 16:56:22.582151 11041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:22.575138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:34315 (local address 127.0.0.1:58222)
0808 16:56:22.575273 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:22.575309 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 16:56:22.575718 (+ 409us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:56:22.575897 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:56:22.575900 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:56:22.576109 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:56:22.576283 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:22.576287 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:22.576843 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:22.576844 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:22.577578 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:22.577580 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:22.577867 (+ 287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:22.577868 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:22.577959 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:22.578397 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:56:22.578407 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:56:22.579613 (+ 1206us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:56:22.580879 (+ 1266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:56:22.580885 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:56:22.580887 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:56:22.581082 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:56:22.581298 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:56:22.581298 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:56:22.581299 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:56:22.581394 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:56:22.581614 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:56:22.581621 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:56:22.581739 (+ 118us) client_negotiation.cc:769] Sending connection context
0808 16:56:22.581829 (+ 90us) client_negotiation.cc:240] Negotiation successful
0808 16:56:22.581964 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":67,"threads_started":1}
I20260808 16:56:22.582686 11039 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.24.61" port: 40469 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.8.24.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.60@KRBTEST.COM" created.
Aug 08 16:56:22 dist-test-slave-6zbq krb5kdc[10814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208182, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.61@KRBTEST.COM for kudu/127.8.24.61@KRBTEST.COM
I20260808 16:56:22.591408 11041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:22.582885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.61:40469 (local address 127.0.0.1:58048)
0808 16:56:22.582922 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:22.582932 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 16:56:22.582969 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:56:22.583195 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:56:22.583197 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:56:22.583328 (+ 131us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:56:22.583382 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:22.583383 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:22.584567 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:22.584568 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:22.585037 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:22.585039 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:22.585623 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:22.585624 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:22.585667 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:22.585986 (+ 319us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:56:22.585990 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:56:22.588770 (+ 2780us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:56:22.590453 (+ 1683us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:56:22.590454 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:56:22.590454 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:56:22.590632 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:56:22.590818 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:56:22.590819 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:56:22.590819 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:56:22.590882 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:56:22.591096 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:56:22.591097 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:56:22.591138 (+ 41us) client_negotiation.cc:769] Sending connection context
0808 16:56:22.591208 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 16:56:22.591306 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23}
I20260808 16:56:22.591467 11044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:22.582950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58048 (local address 127.8.24.61:40469)
0808 16:56:22.583071 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:22.583075 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:22.583090 (+ 15us) server_negotiation.cc:407] Connection header received
0808 16:56:22.583108 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:22.583109 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:22.583118 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:22.583169 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:22.583466 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:22.584497 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:22.585111 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:22.585555 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:22.585630 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:22.588842 (+ 3212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:22.588849 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:22.588850 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:56:22.588856 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:22.590393 (+ 1537us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:22.590704 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:22.590707 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:22.590707 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:22.590759 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:22.590934 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:22.590934 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:22.590934 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:22.591043 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:22.591096 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:22.591228 (+ 132us) server_negotiation.cc:299] Negotiation successful
0808 16:56:22.591359 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":57,"server-negotiator.queue_time_us":92,"thread_start_us":70,"threads_started":1}
I20260808 16:56:22.592139 11039 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.24.60" port: 39733 } has no permanent_uuid. Determining permanent_uuid...
I20260808 16:56:22.592424 11041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:22.592307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.60:39733 (local address 127.0.0.1:59912)
0808 16:56:22.592351 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:22.592400 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.60:39733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260808 16:56:22.592546 10973 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.24.60:39733: connect: Connection refused (error 111)
I20260808 16:56:22.592901 11041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:22.592812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.60:39733 (local address 127.0.0.1:59914)
0808 16:56:22.592853 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:22.592881 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.60:39733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260808 16:56:22.592994 11039 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.24.60:39733: Network error: Client connection negotiation failed: client connection to 127.8.24.60:39733: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.60.keytab.
Entry for principal kudu/127.8.24.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.60.keytab.
I20260808 16:56:22.601049 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786207919384338-8288-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786207919384338-8288-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786207919384338-8288-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786207919384338-8288-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.60:39733
--webserver_interface=127.8.24.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.60.keytab
--principal=kudu/127.8.24.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:46585
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:34315,127.8.24.61:40469,127.8.24.60:39733
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 16:56:22.642195 11039 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.24.60" port: 39733 } attempt: 1
I20260808 16:56:22.642590 11041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:22.642475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.60:39733 (local address 127.0.0.1:59922)
0808 16:56:22.642525 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:22.642561 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.60:39733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260808 16:56:22.642899 11041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:22.642817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.60:39733 (local address 127.0.0.1:59928)
0808 16:56:22.642855 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:22.642880 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.60:39733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260808 16:56:22.642985 11039 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.24.60:39733: Network error: Client connection negotiation failed: client connection to 127.8.24.60:39733: connect: Connection refused (error 111)
I20260808 16:56:22.646915 10934 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.24.61" port: 40469 } attempt: 3
Aug 08 16:56:22 dist-test-slave-6zbq krb5kdc[10814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208182, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for kudu/127.8.24.61@KRBTEST.COM
I20260808 16:56:22.653630 11044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:22.647210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58056 (local address 127.8.24.61:40469)
0808 16:56:22.647250 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:22.647252 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:22.647279 (+ 27us) server_negotiation.cc:407] Connection header received
0808 16:56:22.647333 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:22.647334 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:22.647344 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:22.647399 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:22.647639 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:22.648387 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:22.649016 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:22.649350 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:22.649412 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:22.651101 (+ 1689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:22.651102 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:22.651103 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:56:22.651110 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:22.652427 (+ 1317us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:22.652832 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:22.652833 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:22.652834 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:22.652894 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:22.653162 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:22.653163 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:22.653163 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:22.653292 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:22.653356 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:22.653515 (+ 159us) server_negotiation.cc:299] Negotiation successful
0808 16:56:22.653578 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 16:56:22.653674 10938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:22.647168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.61:40469 (local address 127.0.0.1:58056)
0808 16:56:22.647216 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:22.647227 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 16:56:22.647291 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:56:22.647404 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:56:22.647405 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:56:22.647526 (+ 121us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:56:22.647585 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:22.647587 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:22.648461 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:22.648462 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:22.648955 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:22.648956 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:22.649412 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:22.649413 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:22.649472 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:22.649858 (+ 386us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:56:22.649862 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:56:22.651017 (+ 1155us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:56:22.652534 (+ 1517us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:56:22.652535 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:56:22.652536 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:56:22.652748 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:56:22.652976 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:56:22.652977 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:56:22.652977 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:56:22.653075 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:56:22.653349 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:56:22.653350 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:56:22.653421 (+ 71us) client_negotiation.cc:769] Sending connection context
0808 16:56:22.653491 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 16:56:22.653584 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260808 16:56:22.654067 10934 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.24.60" port: 39733 } has no permanent_uuid. Determining permanent_uuid...
I20260808 16:56:22.654378 10938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:22.654265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.60:39733 (local address 127.0.0.1:59930)
0808 16:56:22.654304 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:22.654335 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.60:39733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260808 16:56:22.654659 10938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:22.654588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.60:39733 (local address 127.0.0.1:59932)
0808 16:56:22.654614 (+ 26us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:22.654638 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.60:39733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
W20260808 16:56:22.654752 10934 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.24.60:39733: Network error: Client connection negotiation failed: client connection to 127.8.24.60:39733: connect: Connection refused (error 111)
W20260808 16:56:22.684252 11047 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 16:56:22.684422 11047 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:56:22.684437 11047 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:56:22.684460 11047 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:56:22.685488 11047 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 16:56:22.685518 11047 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:56:22.685531 11047 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:56:22.685542 11047 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 16:56:22.685559 11047 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 16:56:22.685905 10934 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.24.60" port: 39733 } attempt: 1
I20260808 16:56:22.686195 10938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:22.686102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.60:39733 (local address 127.0.0.1:59942)
0808 16:56:22.686145 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:22.686174 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.60:39733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260808 16:56:22.686514 10938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:22.686447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.60:39733 (local address 127.0.0.1:59956)
0808 16:56:22.686478 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:22.686499 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.60:39733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260808 16:56:22.686594 10934 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.24.60:39733: Network error: Client connection negotiation failed: client connection to 127.8.24.60:39733: connect: Connection refused (error 111)
I20260808 16:56:22.687192 11047 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:46585
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786207919384338-8288-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786207919384338-8288-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:34315,127.8.24.61:40469,127.8.24.60:39733
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.60.keytab
--principal=kudu/127.8.24.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.60:39733
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786207919384338-8288-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.8.24.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11047
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786207919384338-8288-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:56:22.687381 11047 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:56:22.687539 11047 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:56:22.691727 11061 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:22.691732 11059 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:22.691728 11057 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:56:22.691803 11047 server_base.cc:1034] running on GCE node
I20260808 16:56:22.692181 11047 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:56:22.692461 11047 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:56:22.693598 11047 hybrid_clock.cc:648] HybridClock initialized: now 1786208182693580 us; error 26 us; skew 500 ppm
Aug 08 16:56:22 dist-test-slave-6zbq krb5kdc[10814](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208182, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:56:22.695660 11047 init.cc:377] Logged in from keytab as kudu/127.8.24.60@KRBTEST.COM (short username kudu)
I20260808 16:56:22.696051 11047 webserver.cc:460] Webserver started at http://127.8.24.60:37301/ using document root <none> and password file <none>
I20260808 16:56:22.696236 11047 fs_manager.cc:359] Metadata directory not provided
I20260808 16:56:22.696288 11047 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:56:22.696408 11047 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:56:22.697314 11047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786207919384338-8288-0/minicluster-data/master-2/data/instance:
uuid: "52bafb3a7afe44aba9b4d87cec3611ca"
format_stamp: "Formatted at 2026-08-08 16:56:22 on dist-test-slave-6zbq"
server_key: "c4fe408d043a3904ccf9d7366d9bedc6"
server_key_iv: "26260946cf95c822f5a96074c1fc9cca"
server_key_version: "encryptionkey@0"
I20260808 16:56:22.697610 11047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786207919384338-8288-0/minicluster-data/master-2/wal/instance:
uuid: "52bafb3a7afe44aba9b4d87cec3611ca"
format_stamp: "Formatted at 2026-08-08 16:56:22 on dist-test-slave-6zbq"
server_key: "c4fe408d043a3904ccf9d7366d9bedc6"
server_key_iv: "26260946cf95c822f5a96074c1fc9cca"
server_key_version: "encryptionkey@0"
I20260808 16:56:22.698799 11047 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 16:56:22.699385 11071 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:22.699527 11047 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:22.699604 11047 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786207919384338-8288-0/minicluster-data/master-2/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786207919384338-8288-0/minicluster-data/master-2/wal
uuid: "52bafb3a7afe44aba9b4d87cec3611ca"
format_stamp: "Formatted at 2026-08-08 16:56:22 on dist-test-slave-6zbq"
server_key: "c4fe408d043a3904ccf9d7366d9bedc6"
server_key_iv: "26260946cf95c822f5a96074c1fc9cca"
server_key_version: "encryptionkey@0"
I20260808 16:56:22.699661 11047 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786207919384338-8288-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786207919384338-8288-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786207919384338-8288-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:56:22.707701 11047 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:56:22.709239 11047 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:56:22.709364 11047 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:56:22.711129 11039 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.24.60" port: 39733 } attempt: 2
I20260808 16:56:22.711457 11041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:22.711353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.60:39733 (local address 127.0.0.1:59966)
0808 16:56:22.711392 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:22.711427 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.60:39733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260808 16:56:22.711828 11041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:22.711732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.60:39733 (local address 127.0.0.1:59970)
0808 16:56:22.711780 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:22.711807 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.60:39733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260808 16:56:22.711951 11039 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.24.60:39733: Network error: Client connection negotiation failed: client connection to 127.8.24.60:39733: connect: Connection refused (error 111)
I20260808 16:56:22.713021 11047 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.60:39733
I20260808 16:56:22.713063 11154 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.60:39733 every 8 connection(s)
I20260808 16:56:22.713337 11047 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786207919384338-8288-0/minicluster-data/master-2/data/info.pb
I20260808 16:56:22.714172 11156 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:56:22.715005 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 11047
I20260808 16:56:22.715090 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786207919384338-8288-0/minicluster-data/master-2/wal/instance
I20260808 16:56:22.715241 8288 external_mini_cluster.cc:1428] Setting key eed46aa72e10132ee6d3fd1c47b1c7ec
I20260808 16:56:22.715572 11156 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.24.62" port: 34315 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 16:56:22 dist-test-slave-6zbq krb5kdc[10814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208182, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.60@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
Aug 08 16:56:22 dist-test-slave-6zbq krb5kdc[10814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208182, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.24.60@KRBTEST.COM
I20260808 16:56:22.722862 10939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:22.716147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58228 (local address 127.8.24.62:34315)
0808 16:56:22.716193 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:22.716195 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:22.716352 (+ 157us) server_negotiation.cc:407] Connection header received
0808 16:56:22.716427 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:22.716428 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:22.716438 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:22.716476 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:22.716958 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:22.717406 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:22.718204 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:22.718445 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:22.718494 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:22.720350 (+ 1856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:22.720351 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:22.720352 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:56:22.720358 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:22.721537 (+ 1179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:22.721938 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:22.721939 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:22.721940 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:22.721992 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:22.722199 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:22.722200 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:22.722201 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:22.722325 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:22.722403 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:22.722715 (+ 312us) server_negotiation.cc:299] Negotiation successful
0808 16:56:22.722786 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 16:56:22.722992 11163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:22.716121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:34315 (local address 127.0.0.1:58228)
0808 16:56:22.716262 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:22.716291 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0808 16:56:22.716357 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:56:22.716504 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:56:22.716506 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:56:22.716735 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:56:22.716876 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:22.716879 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:22.717469 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:22.717470 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:22.718128 (+ 658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:22.718130 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:22.718537 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:22.718538 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:22.718631 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:22.719010 (+ 379us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:56:22.719019 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:56:22.720276 (+ 1257us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:56:22.721612 (+ 1336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:56:22.721619 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:56:22.721620 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:56:22.721873 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:56:22.722043 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:56:22.722044 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:56:22.722044 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:56:22.722145 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:56:22.722431 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:56:22.722438 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:56:22.722580 (+ 142us) client_negotiation.cc:769] Sending connection context
0808 16:56:22.722678 (+ 98us) client_negotiation.cc:240] Negotiation successful
0808 16:56:22.722827 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":60,"threads_started":1}
I20260808 16:56:22.723451 11156 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.24.61" port: 40469 } has no permanent_uuid. Determining permanent_uuid...
I20260808 16:56:22.723568 11162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:22.715658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59972 (local address 127.8.24.60:39733)
0808 16:56:22.715799 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:22.715805 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:22.715828 (+ 23us) server_negotiation.cc:407] Connection header received
0808 16:56:22.716305 (+ 477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:22.716313 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:22.716359 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:22.716446 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:22.717020 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:22.717750 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:22.718465 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:22.718759 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:22.718816 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:22.720801 (+ 1985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:22.720808 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:22.720808 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:56:22.720814 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:22.722201 (+ 1387us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:22.722551 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:22.722553 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:22.722553 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:22.722599 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:22.722774 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:22.722775 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:22.722776 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:22.722926 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:22.723035 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:22.723421 (+ 386us) server_negotiation.cc:299] Negotiation successful
0808 16:56:22.723488 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97,"thread_start_us":58,"threads_started":1}
Aug 08 16:56:22 dist-test-slave-6zbq krb5kdc[10814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208182, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.60@KRBTEST.COM for kudu/127.8.24.61@KRBTEST.COM
Aug 08 16:56:22 dist-test-slave-6zbq krb5kdc[10814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208182, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260808 16:56:22.729286 11044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:22.723710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58072 (local address 127.8.24.61:40469)
0808 16:56:22.723766 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:22.723768 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:22.723799 (+ 31us) server_negotiation.cc:407] Connection header received
0808 16:56:22.723857 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:22.723858 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:22.723868 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:22.723926 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:22.724164 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:22.724621 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:22.725209 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:22.725421 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:22.725473 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:22.727075 (+ 1602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:22.727076 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:22.727077 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:56:22.727084 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:22.728364 (+ 1280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:22.728633 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:22.728634 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:22.728635 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:22.728692 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:22.728874 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:22.728875 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:22.728875 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:22.728970 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:22.729020 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:22.729157 (+ 137us) server_negotiation.cc:299] Negotiation successful
0808 16:56:22.729232 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 16:56:22.729286 11163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:22.723650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.61:40469 (local address 127.0.0.1:58072)
0808 16:56:22.723757 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:22.723767 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 16:56:22.723811 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:56:22.723929 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:56:22.723929 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:56:22.724057 (+ 128us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:56:22.724096 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:22.724097 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:22.724686 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:22.724687 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:22.725142 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:22.725144 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:22.725480 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:22.725481 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:22.725538 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:22.725871 (+ 333us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:56:22.725875 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:56:22.727001 (+ 1126us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:56:22.728441 (+ 1440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:56:22.728442 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:56:22.728442 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:56:22.728583 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:56:22.728761 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:56:22.728762 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:56:22.728762 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:56:22.728817 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:56:22.729026 (+ 209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:56:22.729027 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:56:22.729067 (+ 40us) client_negotiation.cc:769] Sending connection context
0808 16:56:22.729119 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 16:56:22.729196 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":85}
I20260808 16:56:22.729573 11156 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.24.60" port: 39733 } has no permanent_uuid. Determining permanent_uuid...
I20260808 16:56:22.730795 10939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:22.724264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58230 (local address 127.8.24.62:34315)
0808 16:56:22.724296 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:22.724298 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:22.724340 (+ 42us) server_negotiation.cc:407] Connection header received
0808 16:56:22.724402 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:22.724403 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:22.724411 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:22.724448 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:22.724868 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:22.725282 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:22.726010 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:22.726205 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:22.726265 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:22.728415 (+ 2150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:22.728416 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:22.728416 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:56:22.728422 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:22.729820 (+ 1398us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:22.730166 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:22.730167 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:22.730167 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:22.730222 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:22.730427 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:22.730427 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:22.730428 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:22.730518 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:22.730612 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:22.730685 (+ 73us) server_negotiation.cc:299] Negotiation successful
0808 16:56:22.730735 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
Aug 08 16:56:22 dist-test-slave-6zbq krb5kdc[10814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208182, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.60@KRBTEST.COM for kudu/127.8.24.60@KRBTEST.COM
Aug 08 16:56:22 dist-test-slave-6zbq krb5kdc[10814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208182, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.24.61@KRBTEST.COM
I20260808 16:56:22.735340 11163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:22.729767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.60:39733 (local address 127.0.0.1:59980)
0808 16:56:22.729847 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:22.729855 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 16:56:22.729896 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:56:22.730005 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:56:22.730005 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:56:22.730098 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:56:22.730120 (+ 22us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:22.730121 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:22.730866 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:22.730867 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:22.731426 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:22.731428 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:22.731844 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:22.731845 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:22.731881 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:22.732207 (+ 326us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:56:22.732211 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:56:22.733164 (+ 953us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:56:22.734522 (+ 1358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:56:22.734523 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:56:22.734523 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:56:22.734659 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:56:22.734828 (+ 169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:56:22.734829 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:56:22.734829 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:56:22.734867 (+ 38us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:56:22.735068 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:56:22.735069 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:56:22.735104 (+ 35us) client_negotiation.cc:769] Sending connection context
0808 16:56:22.735160 (+ 56us) client_negotiation.cc:240] Negotiation successful
0808 16:56:22.735249 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67}
I20260808 16:56:22.735394 11162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:22.729812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59980 (local address 127.8.24.60:39733)
0808 16:56:22.729844 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:22.729846 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:22.729884 (+ 38us) server_negotiation.cc:407] Connection header received
0808 16:56:22.729937 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:22.729938 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:22.729946 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:22.730002 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:22.730179 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:22.730794 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:22.731484 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:22.731782 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:22.731846 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:22.733232 (+ 1386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:22.733233 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:22.733233 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:56:22.733247 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:22.734457 (+ 1210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:22.734716 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:22.734717 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:22.734718 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:22.734771 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:22.734901 (+ 130us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:22.734902 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:22.734902 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:22.735007 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:22.735059 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:22.735187 (+ 128us) server_negotiation.cc:299] Negotiation successful
0808 16:56:22.735296 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":53,"server-negotiator.queue_time_us":17}
I20260808 16:56:22.736753 11156 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52bafb3a7afe44aba9b4d87cec3611ca: Bootstrap starting.
I20260808 16:56:22.737439 11044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:22.731394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58074 (local address 127.8.24.61:40469)
0808 16:56:22.731428 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:22.731429 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:22.731476 (+ 47us) server_negotiation.cc:407] Connection header received
0808 16:56:22.731543 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:22.731544 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:22.731552 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:22.731587 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:22.732070 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:22.732535 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:22.733166 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:22.733361 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:22.733416 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:22.735292 (+ 1876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:22.735293 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:22.735293 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:56:22.735300 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:22.736541 (+ 1241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:22.736822 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:22.736823 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:22.736824 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:22.736875 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:22.737031 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:22.737032 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:22.737032 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:22.737128 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:22.737267 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:22.737304 (+ 37us) server_negotiation.cc:299] Negotiation successful
0808 16:56:22.737364 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260808 16:56:22.737435 11156 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 52bafb3a7afe44aba9b4d87cec3611ca: Neither blocks nor log segments found. Creating new log.
I20260808 16:56:22.737713 11156 log.cc:824] T 00000000000000000000000000000000 P 52bafb3a7afe44aba9b4d87cec3611ca: Log is configured to *not* fsync() on all Append() calls
I20260808 16:56:22.738484 11156 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52bafb3a7afe44aba9b4d87cec3611ca: No bootstrap required, opened a new log
I20260808 16:56:22.738740 8288 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 16:56:22.739889 11156 raft_consensus.cc:348] T 00000000000000000000000000000000 P 52bafb3a7afe44aba9b4d87cec3611ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67c8331b5fa346e0860f71d6d8e5940a" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 34315 } } peers { permanent_uuid: "b90bb81c60a943daa7e1639eb2785c8c" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 40469 } } peers { permanent_uuid: "52bafb3a7afe44aba9b4d87cec3611ca" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 39733 } }
I20260808 16:56:22.739997 11156 raft_consensus.cc:729] T 00000000000000000000000000000000 P 52bafb3a7afe44aba9b4d87cec3611ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52bafb3a7afe44aba9b4d87cec3611ca, State: Initialized, Role: FOLLOWER
I20260808 16:56:22.740069 11156 consensus_queue.cc:260] T 00000000000000000000000000000000 P 52bafb3a7afe44aba9b4d87cec3611ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67c8331b5fa346e0860f71d6d8e5940a" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 34315 } } peers { permanent_uuid: "b90bb81c60a943daa7e1639eb2785c8c" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 40469 } } peers { permanent_uuid: "52bafb3a7afe44aba9b4d87cec3611ca" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 39733 } }
I20260808 16:56:22.740298 11167 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52bafb3a7afe44aba9b4d87cec3611ca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67c8331b5fa346e0860f71d6d8e5940a" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 34315 } } peers { permanent_uuid: "b90bb81c60a943daa7e1639eb2785c8c" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 40469 } } peers { permanent_uuid: "52bafb3a7afe44aba9b4d87cec3611ca" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 39733 } } }
I20260808 16:56:22.739671 10891 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "67c8331b5fa346e0860f71d6d8e5940a"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58230
I20260808 16:56:22.740350 11156 sys_catalog.cc:564] T 00000000000000000000000000000000 P 52bafb3a7afe44aba9b4d87cec3611ca [sys.catalog]: configured and running, proceeding with master startup.
I20260808 16:56:22.740396 11167 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52bafb3a7afe44aba9b4d87cec3611ca [sys.catalog]: This master's current role is: FOLLOWER
W20260808 16:56:22.741972 11184 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 52bafb3a7afe44aba9b4d87cec3611ca: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 16:56:22.742023 11184 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 16:56:22.742228 11047 master_runner.cc:186] Error getting master registration for 127.8.24.62:34315: OK, instance_id { permanent_uuid: "67c8331b5fa346e0860f71d6d8e5940a" instance_seqno: 1786208182423686 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 16:56:22.744975 11047 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 16:56:22.761698 10934 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.24.60" port: 39733 } attempt: 2
Aug 08 16:56:22 dist-test-slave-6zbq krb5kdc[10814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208182, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for kudu/127.8.24.60@KRBTEST.COM
I20260808 16:56:22.768548 11162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:22.761990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59994 (local address 127.8.24.60:39733)
0808 16:56:22.762039 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:22.762040 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:22.762049 (+ 9us) server_negotiation.cc:407] Connection header received
0808 16:56:22.762069 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:22.762070 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:22.762076 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:22.762128 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:22.762456 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:22.763140 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:22.764062 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:22.764348 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:22.764399 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:22.766241 (+ 1842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:22.766242 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:22.766242 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:56:22.766249 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:22.767506 (+ 1257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:22.767844 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:22.767845 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:22.767845 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:22.767894 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:22.768047 (+ 153us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:22.768048 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:22.768049 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:22.768201 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:22.768264 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:22.768408 (+ 144us) server_negotiation.cc:299] Negotiation successful
0808 16:56:22.768467 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260808 16:56:22.768553 10938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:22.761912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.60:39733 (local address 127.0.0.1:59994)
0808 16:56:22.761974 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:22.761984 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 16:56:22.762028 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:56:22.762147 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:56:22.762147 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:56:22.762302 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:56:22.762392 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:22.762395 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:22.763207 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:22.763208 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:22.763991 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:22.763993 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:22.764403 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:22.764404 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:22.764460 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:22.764851 (+ 391us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:56:22.764855 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:56:22.766177 (+ 1322us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:56:22.767578 (+ 1401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:56:22.767579 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:56:22.767580 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:56:22.767779 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:56:22.767945 (+ 166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:56:22.767946 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:56:22.767946 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:56:22.768001 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:56:22.768279 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:56:22.768280 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:56:22.768321 (+ 41us) client_negotiation.cc:769] Sending connection context
0808 16:56:22.768378 (+ 57us) client_negotiation.cc:240] Negotiation successful
0808 16:56:22.768464 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260808 16:56:22.769591 10934 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 67c8331b5fa346e0860f71d6d8e5940a: Bootstrap starting.
I20260808 16:56:22.770103 10934 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 67c8331b5fa346e0860f71d6d8e5940a: Neither blocks nor log segments found. Creating new log.
I20260808 16:56:22.770373 10934 log.cc:824] T 00000000000000000000000000000000 P 67c8331b5fa346e0860f71d6d8e5940a: Log is configured to *not* fsync() on all Append() calls
I20260808 16:56:22.771008 10934 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 67c8331b5fa346e0860f71d6d8e5940a: No bootstrap required, opened a new log
I20260808 16:56:22.771946 10934 raft_consensus.cc:348] T 00000000000000000000000000000000 P 67c8331b5fa346e0860f71d6d8e5940a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67c8331b5fa346e0860f71d6d8e5940a" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 34315 } } peers { permanent_uuid: "b90bb81c60a943daa7e1639eb2785c8c" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 40469 } } peers { permanent_uuid: "52bafb3a7afe44aba9b4d87cec3611ca" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 39733 } }
I20260808 16:56:22.772044 10934 raft_consensus.cc:729] T 00000000000000000000000000000000 P 67c8331b5fa346e0860f71d6d8e5940a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67c8331b5fa346e0860f71d6d8e5940a, State: Initialized, Role: FOLLOWER
I20260808 16:56:22.772114 10934 consensus_queue.cc:260] T 00000000000000000000000000000000 P 67c8331b5fa346e0860f71d6d8e5940a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67c8331b5fa346e0860f71d6d8e5940a" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 34315 } } peers { permanent_uuid: "b90bb81c60a943daa7e1639eb2785c8c" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 40469 } } peers { permanent_uuid: "52bafb3a7afe44aba9b4d87cec3611ca" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 39733 } }
I20260808 16:56:22.772339 11188 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67c8331b5fa346e0860f71d6d8e5940a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67c8331b5fa346e0860f71d6d8e5940a" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 34315 } } peers { permanent_uuid: "b90bb81c60a943daa7e1639eb2785c8c" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 40469 } } peers { permanent_uuid: "52bafb3a7afe44aba9b4d87cec3611ca" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 39733 } } }
I20260808 16:56:22.772418 10891 raft_consensus.cc:482] T 00000000000000000000000000000000 P 67c8331b5fa346e0860f71d6d8e5940a [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260808 16:56:22.772389 10934 sys_catalog.cc:564] T 00000000000000000000000000000000 P 67c8331b5fa346e0860f71d6d8e5940a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 16:56:22.772537 10891 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 67c8331b5fa346e0860f71d6d8e5940a [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:56:22.773281 10891 raft_consensus.cc:504] T 00000000000000000000000000000000 P 67c8331b5fa346e0860f71d6d8e5940a [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67c8331b5fa346e0860f71d6d8e5940a" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 34315 } } peers { permanent_uuid: "b90bb81c60a943daa7e1639eb2785c8c" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 40469 } } peers { permanent_uuid: "52bafb3a7afe44aba9b4d87cec3611ca" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 39733 } }
I20260808 16:56:22.773896 10891 leader_election.cc:288] T 00000000000000000000000000000000 P 67c8331b5fa346e0860f71d6d8e5940a [CANDIDATE]: Term 1 election: Requested vote from peers b90bb81c60a943daa7e1639eb2785c8c (127.8.24.61:40469), 52bafb3a7afe44aba9b4d87cec3611ca (127.8.24.60:39733)
W20260808 16:56:22.774091 11203 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 67c8331b5fa346e0860f71d6d8e5940a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 16:56:22.774199 11203 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 16:56:22.774328 11116 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "67c8331b5fa346e0860f71d6d8e5940a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "52bafb3a7afe44aba9b4d87cec3611ca"
I20260808 16:56:22.774459 11116 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 52bafb3a7afe44aba9b4d87cec3611ca [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:56:22.772450 11188 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67c8331b5fa346e0860f71d6d8e5940a [sys.catalog]: This master's current role is: FOLLOWER
I20260808 16:56:22.775238 11116 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 52bafb3a7afe44aba9b4d87cec3611ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67c8331b5fa346e0860f71d6d8e5940a in term 1.
I20260808 16:56:22.776026 10831 master_runner.cc:186] Error getting master registration for 127.8.24.61:40469: OK, instance_id { permanent_uuid: "b90bb81c60a943daa7e1639eb2785c8c" instance_seqno: 1786208182571029 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 16:56:22.778126 10831 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 16:56:22.778467 10856 leader_election.cc:302] T 00000000000000000000000000000000 P 67c8331b5fa346e0860f71d6d8e5940a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bafb3a7afe44aba9b4d87cec3611ca, 67c8331b5fa346e0860f71d6d8e5940a; no voters:
I20260808 16:56:22.778579 11188 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 67c8331b5fa346e0860f71d6d8e5940a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:56:22.778726 11188 raft_consensus.cc:686] T 00000000000000000000000000000000 P 67c8331b5fa346e0860f71d6d8e5940a [term 1 LEADER]: Becoming Leader. State: Replica: 67c8331b5fa346e0860f71d6d8e5940a, State: Running, Role: LEADER
I20260808 16:56:22.778816 11188 consensus_queue.cc:237] T 00000000000000000000000000000000 P 67c8331b5fa346e0860f71d6d8e5940a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67c8331b5fa346e0860f71d6d8e5940a" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 34315 } } peers { permanent_uuid: "b90bb81c60a943daa7e1639eb2785c8c" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 40469 } } peers { permanent_uuid: "52bafb3a7afe44aba9b4d87cec3611ca" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 39733 } }
I20260808 16:56:22.779181 11188 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67c8331b5fa346e0860f71d6d8e5940a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 67c8331b5fa346e0860f71d6d8e5940a. Latest consensus state: current_term: 1 leader_uuid: "67c8331b5fa346e0860f71d6d8e5940a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67c8331b5fa346e0860f71d6d8e5940a" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 34315 } } peers { permanent_uuid: "b90bb81c60a943daa7e1639eb2785c8c" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 40469 } } peers { permanent_uuid: "52bafb3a7afe44aba9b4d87cec3611ca" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 39733 } } }
I20260808 16:56:22.779258 11188 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67c8331b5fa346e0860f71d6d8e5940a [sys.catalog]: This master's current role is: LEADER
I20260808 16:56:22.783387 11223 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 16:56:22.783619 11223 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 16:56:22.786731 11162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:22.775134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59998 (local address 127.8.24.60:39733)
0808 16:56:22.775175 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:22.775177 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:22.776087 (+ 910us) server_negotiation.cc:407] Connection header received
0808 16:56:22.776233 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:22.776233 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:22.776255 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:22.776283 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:22.777586 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:22.778292 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:22.779792 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:22.780082 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:22.780118 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:22.783356 (+ 3238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:22.783357 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:22.783357 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:56:22.783364 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:22.784958 (+ 1594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:22.785536 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:22.785537 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:22.785537 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:22.785674 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:22.785988 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:22.785989 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:22.785989 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:22.786153 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:22.786219 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:22.786570 (+ 351us) server_negotiation.cc:299] Negotiation successful
0808 16:56:22.786643 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 16:56:22.788188 11116 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 52bafb3a7afe44aba9b4d87cec3611ca [term 1 FOLLOWER]: Refusing update from remote peer 67c8331b5fa346e0860f71d6d8e5940a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:56:22.788542 11218 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 67c8331b5fa346e0860f71d6d8e5940a [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bafb3a7afe44aba9b4d87cec3611ca" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 39733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:56:22.789685 11167 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52bafb3a7afe44aba9b4d87cec3611ca [sys.catalog]: SysCatalogTable state changed. Reason: New leader 67c8331b5fa346e0860f71d6d8e5940a. Latest consensus state: current_term: 1 leader_uuid: "67c8331b5fa346e0860f71d6d8e5940a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67c8331b5fa346e0860f71d6d8e5940a" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 34315 } } peers { permanent_uuid: "b90bb81c60a943daa7e1639eb2785c8c" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 40469 } } peers { permanent_uuid: "52bafb3a7afe44aba9b4d87cec3611ca" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 39733 } } }
I20260808 16:56:22.789875 11167 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52bafb3a7afe44aba9b4d87cec3611ca [sys.catalog]: This master's current role is: FOLLOWER
I20260808 16:56:22.793944 11044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:22.775865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58078 (local address 127.8.24.61:40469)
0808 16:56:22.775898 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:22.775900 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:22.782471 (+ 6571us) server_negotiation.cc:407] Connection header received
0808 16:56:22.782572 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:22.782573 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:22.782583 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:22.782613 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:22.784406 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:22.785102 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:22.786068 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:22.786391 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:22.786431 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:22.790029 (+ 3598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:22.790030 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:22.790031 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:56:22.790038 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:22.792261 (+ 2223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:22.792677 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:22.792678 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:22.792679 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:22.792737 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:22.793024 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:22.793025 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:22.793025 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:22.793169 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:22.793734 (+ 565us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:22.793778 (+ 44us) server_negotiation.cc:299] Negotiation successful
0808 16:56:22.793853 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260808 16:56:22.794159 10939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:22.775981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58244 (local address 127.8.24.62:34315)
0808 16:56:22.776010 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:22.776012 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:22.778178 (+ 2166us) server_negotiation.cc:407] Connection header received
0808 16:56:22.778199 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:22.778199 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:22.778208 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:22.778263 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:22.780191 (+ 1928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:22.780867 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:22.781862 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:22.782137 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:22.782178 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:22.790324 (+ 8146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:22.790325 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:22.790326 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:56:22.790332 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:22.792246 (+ 1914us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:22.793047 (+ 801us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:22.793048 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:22.793048 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:22.793099 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:22.793592 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:22.793593 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:22.793594 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:22.793719 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:22.793803 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:22.793994 (+ 191us) server_negotiation.cc:299] Negotiation successful
0808 16:56:22.794065 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13}
I20260808 16:56:22.799547 11188 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67c8331b5fa346e0860f71d6d8e5940a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "67c8331b5fa346e0860f71d6d8e5940a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67c8331b5fa346e0860f71d6d8e5940a" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 34315 } } peers { permanent_uuid: "b90bb81c60a943daa7e1639eb2785c8c" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 40469 } } peers { permanent_uuid: "52bafb3a7afe44aba9b4d87cec3611ca" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 39733 } } }
I20260808 16:56:22.799630 11188 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67c8331b5fa346e0860f71d6d8e5940a [sys.catalog]: This master's current role is: LEADER
I20260808 16:56:22.799798 11223 catalog_manager.cc:1347] Generated new cluster ID: cad9d2ab1e66487a8b759b5c083a6db8
I20260808 16:56:22.799845 11223 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 16:56:22.800064 11167 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52bafb3a7afe44aba9b4d87cec3611ca [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "67c8331b5fa346e0860f71d6d8e5940a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67c8331b5fa346e0860f71d6d8e5940a" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 34315 } } peers { permanent_uuid: "b90bb81c60a943daa7e1639eb2785c8c" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 40469 } } peers { permanent_uuid: "52bafb3a7afe44aba9b4d87cec3611ca" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 39733 } } }
I20260808 16:56:22.800149 11167 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52bafb3a7afe44aba9b4d87cec3611ca [sys.catalog]: This master's current role is: FOLLOWER
I20260808 16:56:22.800436 11225 mvcc.cc:205] Tried to move back new op lower bound from 7316308716697595904 to 7316308716662644736. Current Snapshot: MvccSnapshot[applied={T|T < 7316308716697595904}]
I20260808 16:56:22.803076 11039 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.24.60" port: 39733 } attempt: 3
Aug 08 16:56:22 dist-test-slave-6zbq krb5kdc[10814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208182, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.61@KRBTEST.COM for kudu/127.8.24.60@KRBTEST.COM
I20260808 16:56:22.810076 11041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:22.803292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.60:39733 (local address 127.0.0.1:60004)
0808 16:56:22.803331 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:22.803341 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 16:56:22.803380 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:56:22.803515 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:56:22.803516 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:56:22.803667 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:56:22.803699 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:22.803701 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:22.804538 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:22.804539 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:22.805269 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:22.805271 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:22.805718 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:22.805720 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:22.805769 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:22.806105 (+ 336us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:56:22.806110 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:56:22.807520 (+ 1410us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:56:22.809097 (+ 1577us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:56:22.809098 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:56:22.809099 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:56:22.809292 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:56:22.809492 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:56:22.809493 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:56:22.809493 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:56:22.809546 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:56:22.809817 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:56:22.809818 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:56:22.809857 (+ 39us) client_negotiation.cc:769] Sending connection context
0808 16:56:22.809912 (+ 55us) client_negotiation.cc:240] Negotiation successful
0808 16:56:22.809995 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22}
I20260808 16:56:22.810410 11162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:22.803383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60004 (local address 127.8.24.60:39733)
0808 16:56:22.803417 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:22.803418 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:22.803425 (+ 7us) server_negotiation.cc:407] Connection header received
0808 16:56:22.803443 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:22.803444 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:22.803451 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:22.803511 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:22.803757 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:22.804464 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:22.805352 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:22.805630 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:22.805677 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:22.807591 (+ 1914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:22.807600 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:22.807600 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:56:22.807607 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:22.809006 (+ 1399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:22.809376 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:22.809378 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:22.809378 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:22.809438 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:22.809622 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:22.809623 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:22.809623 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:22.809739 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:22.809792 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:22.810252 (+ 460us) server_negotiation.cc:299] Negotiation successful
0808 16:56:22.810320 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 16:56:22.810992 11223 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 16:56:22.811663 11223 catalog_manager.cc:1504] Loading token signing keys...
I20260808 16:56:22.811758 11039 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b90bb81c60a943daa7e1639eb2785c8c: Bootstrap starting.
I20260808 16:56:22.812389 11039 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b90bb81c60a943daa7e1639eb2785c8c: Neither blocks nor log segments found. Creating new log.
I20260808 16:56:22.812666 11039 log.cc:824] T 00000000000000000000000000000000 P b90bb81c60a943daa7e1639eb2785c8c: Log is configured to *not* fsync() on all Append() calls
I20260808 16:56:22.819060 11039 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b90bb81c60a943daa7e1639eb2785c8c: No bootstrap required, opened a new log
I20260808 16:56:22.820969 11039 raft_consensus.cc:348] T 00000000000000000000000000000000 P b90bb81c60a943daa7e1639eb2785c8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67c8331b5fa346e0860f71d6d8e5940a" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 34315 } } peers { permanent_uuid: "b90bb81c60a943daa7e1639eb2785c8c" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 40469 } } peers { permanent_uuid: "52bafb3a7afe44aba9b4d87cec3611ca" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 39733 } }
I20260808 16:56:22.821089 11039 raft_consensus.cc:729] T 00000000000000000000000000000000 P b90bb81c60a943daa7e1639eb2785c8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b90bb81c60a943daa7e1639eb2785c8c, State: Initialized, Role: FOLLOWER
I20260808 16:56:22.821203 11039 consensus_queue.cc:260] T 00000000000000000000000000000000 P b90bb81c60a943daa7e1639eb2785c8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67c8331b5fa346e0860f71d6d8e5940a" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 34315 } } peers { permanent_uuid: "b90bb81c60a943daa7e1639eb2785c8c" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 40469 } } peers { permanent_uuid: "52bafb3a7afe44aba9b4d87cec3611ca" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 39733 } }
I20260808 16:56:22.821488 11039 sys_catalog.cc:564] T 00000000000000000000000000000000 P b90bb81c60a943daa7e1639eb2785c8c [sys.catalog]: configured and running, proceeding with master startup.
I20260808 16:56:22.823249 11005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "67c8331b5fa346e0860f71d6d8e5940a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "b90bb81c60a943daa7e1639eb2785c8c"
I20260808 16:56:22.823354 11005 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b90bb81c60a943daa7e1639eb2785c8c [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:56:22.824121 11005 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b90bb81c60a943daa7e1639eb2785c8c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67c8331b5fa346e0860f71d6d8e5940a in term 1.
I20260808 16:56:22.824488 11004 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b90bb81c60a943daa7e1639eb2785c8c [term 1 FOLLOWER]: Refusing update from remote peer 67c8331b5fa346e0860f71d6d8e5940a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:56:22.824772 11229 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 67c8331b5fa346e0860f71d6d8e5940a [LEADER]: Connected to new peer: Peer: permanent_uuid: "b90bb81c60a943daa7e1639eb2785c8c" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 40469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:56:22.828889 11234 sys_catalog.cc:455] T 00000000000000000000000000000000 P b90bb81c60a943daa7e1639eb2785c8c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "67c8331b5fa346e0860f71d6d8e5940a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67c8331b5fa346e0860f71d6d8e5940a" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 34315 } } peers { permanent_uuid: "b90bb81c60a943daa7e1639eb2785c8c" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 40469 } } peers { permanent_uuid: "52bafb3a7afe44aba9b4d87cec3611ca" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 39733 } } }
I20260808 16:56:22.828979 11234 sys_catalog.cc:458] T 00000000000000000000000000000000 P b90bb81c60a943daa7e1639eb2785c8c [sys.catalog]: This master's current role is: FOLLOWER
I20260808 16:56:22.829023 11234 sys_catalog.cc:455] T 00000000000000000000000000000000 P b90bb81c60a943daa7e1639eb2785c8c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 67c8331b5fa346e0860f71d6d8e5940a. Latest consensus state: current_term: 1 leader_uuid: "67c8331b5fa346e0860f71d6d8e5940a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67c8331b5fa346e0860f71d6d8e5940a" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 34315 } } peers { permanent_uuid: "b90bb81c60a943daa7e1639eb2785c8c" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 40469 } } peers { permanent_uuid: "52bafb3a7afe44aba9b4d87cec3611ca" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 39733 } } }
I20260808 16:56:22.829074 11234 sys_catalog.cc:458] T 00000000000000000000000000000000 P b90bb81c60a943daa7e1639eb2785c8c [sys.catalog]: This master's current role is: FOLLOWER
I20260808 16:56:22.829109 11234 sys_catalog.cc:455] T 00000000000000000000000000000000 P b90bb81c60a943daa7e1639eb2785c8c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "67c8331b5fa346e0860f71d6d8e5940a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67c8331b5fa346e0860f71d6d8e5940a" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 34315 } } peers { permanent_uuid: "b90bb81c60a943daa7e1639eb2785c8c" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 40469 } } peers { permanent_uuid: "52bafb3a7afe44aba9b4d87cec3611ca" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 39733 } } }
I20260808 16:56:22.829144 11234 sys_catalog.cc:458] T 00000000000000000000000000000000 P b90bb81c60a943daa7e1639eb2785c8c [sys.catalog]: This master's current role is: FOLLOWER
I20260808 16:56:22.829866 11223 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 67c8331b5fa346e0860f71d6d8e5940a: Generated new TSK 0
I20260808 16:56:22.830024 11223 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 16:56:22.831061 11258 mvcc.cc:205] Tried to move back new op lower bound from 7316308716789932032 to 7316308716662644736. Current Snapshot: MvccSnapshot[applied={T|T < 7316308716697595904}]
I20260808 16:56:22.832486 11252 catalog_manager.cc:1259] Loaded cluster ID: cad9d2ab1e66487a8b759b5c083a6db8
I20260808 16:56:22.832532 11252 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b90bb81c60a943daa7e1639eb2785c8c: loading cluster ID for follower catalog manager: success
I20260808 16:56:22.833581 11252 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b90bb81c60a943daa7e1639eb2785c8c: acquiring CA information for follower catalog manager: success
W20260808 16:56:22.833707 11252 catalog_manager.cc:873] Not found: no TSK found in the system table: failed to prepare follower catalog manager, will retry
I20260808 16:56:22.834000 11218 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67c8331b5fa346e0860f71d6d8e5940a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "67c8331b5fa346e0860f71d6d8e5940a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67c8331b5fa346e0860f71d6d8e5940a" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 34315 } } peers { permanent_uuid: "b90bb81c60a943daa7e1639eb2785c8c" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 40469 } } peers { permanent_uuid: "52bafb3a7afe44aba9b4d87cec3611ca" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 39733 } } }
I20260808 16:56:22.834080 11218 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67c8331b5fa346e0860f71d6d8e5940a [sys.catalog]: This master's current role is: LEADER
W20260808 16:56:22.858084 11162 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:56:22.856846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60012 (local address 127.8.24.60:39733)
0808 16:56:22.856891 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:22.856893 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:22.857475 (+ 582us) server_negotiation.cc:407] Connection header received
0808 16:56:22.857576 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:22.857577 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:22.857587 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:22.857616 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:22.858028 (+ 412us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60012: BlockingRecv error: recv got EOF from 127.0.0.1:60012 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260808 16:56:22.858812 10939 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:56:22.857212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58248 (local address 127.8.24.62:34315)
0808 16:56:22.857249 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:22.857250 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:22.858321 (+ 1071us) server_negotiation.cc:407] Connection header received
0808 16:56:22.858437 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:22.858438 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:22.858447 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:22.858473 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:22.858761 (+ 288us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58248: BlockingRecv error: recv got EOF from 127.0.0.1:58248 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260808 16:56:22.859753 11044 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:56:22.859128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58090 (local address 127.8.24.61:40469)
0808 16:56:22.859176 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:22.859177 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:22.859289 (+ 112us) server_negotiation.cc:407] Connection header received
0808 16:56:22.859370 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:22.859370 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:22.859380 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:22.859406 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:22.859688 (+ 282us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58090: BlockingRecv error: recv got EOF from 127.0.0.1:58090 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260808 16:56:22.863236 11044 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:56:22.862566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58102 (local address 127.8.24.61:40469)
0808 16:56:22.862607 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:22.862608 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:22.862765 (+ 157us) server_negotiation.cc:407] Connection header received
0808 16:56:22.862859 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:22.862860 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:22.862875 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:22.862901 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:22.863198 (+ 297us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58102: BlockingRecv error: recv got EOF from 127.0.0.1:58102 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260808 16:56:22.865945 11044 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:56:22.865286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58110 (local address 127.8.24.61:40469)
0808 16:56:22.865315 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:22.865317 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:22.865508 (+ 191us) server_negotiation.cc:407] Connection header received
0808 16:56:22.865594 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:22.865595 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:22.865604 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:22.865628 (+ 24us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:22.865907 (+ 279us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58110: BlockingRecv error: recv got EOF from 127.0.0.1:58110 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260808 16:56:22.869626 11044 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:56:22.868970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58116 (local address 127.8.24.61:40469)
0808 16:56:22.869008 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:22.869009 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:22.869200 (+ 191us) server_negotiation.cc:407] Connection header received
0808 16:56:22.869284 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:22.869285 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:22.869294 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:22.869320 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:22.869589 (+ 269us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58116: BlockingRecv error: recv got EOF from 127.0.0.1:58116 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260808 16:56:22.879374 11044 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:56:22.878569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58126 (local address 127.8.24.61:40469)
0808 16:56:22.878602 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:22.878603 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:22.878879 (+ 276us) server_negotiation.cc:407] Connection header received
0808 16:56:22.878972 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:22.878973 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:22.878992 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:22.879021 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:22.879336 (+ 315us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58126: BlockingRecv error: recv got EOF from 127.0.0.1:58126 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260808 16:56:22.889050 11044 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:56:22.888396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58142 (local address 127.8.24.61:40469)
0808 16:56:22.888426 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:22.888428 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:22.888619 (+ 191us) server_negotiation.cc:407] Connection header received
0808 16:56:22.888706 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:22.888707 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:22.888715 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:22.888739 (+ 24us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:22.889000 (+ 261us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58142: BlockingRecv error: recv got EOF from 127.0.0.1:58142 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260808 16:56:22.906558 11044 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:56:22.905989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58158 (local address 127.8.24.61:40469)
0808 16:56:22.906022 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:22.906024 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:22.906119 (+ 95us) server_negotiation.cc:407] Connection header received
0808 16:56:22.906181 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:22.906182 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:22.906190 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:22.906243 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:22.906515 (+ 272us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58158: BlockingRecv error: recv got EOF from 127.0.0.1:58158 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260808 16:56:22.939929 11044 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:56:22.939331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58174 (local address 127.8.24.61:40469)
0808 16:56:22.939371 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:22.939373 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:22.939495 (+ 122us) server_negotiation.cc:407] Connection header received
0808 16:56:22.939551 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:22.939552 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:22.939563 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:22.939609 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:22.939881 (+ 272us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58174: BlockingRecv error: recv got EOF from 127.0.0.1:58174 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260808 16:56:23.005506 11044 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:56:23.004846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58188 (local address 127.8.24.61:40469)
0808 16:56:23.004899 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:23.004901 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:23.005003 (+ 102us) server_negotiation.cc:407] Connection header received
0808 16:56:23.005070 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:23.005071 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:23.005083 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:23.005130 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:23.005454 (+ 324us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58188: BlockingRecv error: recv got EOF from 127.0.0.1:58188 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
W20260808 16:56:23.134964 11044 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:56:23.134411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58196 (local address 127.8.24.61:40469)
0808 16:56:23.134464 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:23.134465 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:23.134552 (+ 87us) server_negotiation.cc:407] Connection header received
0808 16:56:23.134597 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:23.134597 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:23.134607 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:23.134657 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:23.134913 (+ 256us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58196: BlockingRecv error: recv got EOF from 127.0.0.1:58196 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
W20260808 16:56:23.392699 11044 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:56:23.392095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58198 (local address 127.8.24.61:40469)
0808 16:56:23.392154 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:23.392156 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:23.392167 (+ 11us) server_negotiation.cc:407] Connection header received
0808 16:56:23.392257 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:23.392258 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:23.392268 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:23.392315 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:23.392633 (+ 318us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58198: BlockingRecv error: recv got EOF from 127.0.0.1:58198 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 16:56:23.742427 11184 catalog_manager.cc:1259] Loaded cluster ID: cad9d2ab1e66487a8b759b5c083a6db8
I20260808 16:56:23.742486 11184 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 52bafb3a7afe44aba9b4d87cec3611ca: loading cluster ID for follower catalog manager: success
I20260808 16:56:23.743212 11184 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 52bafb3a7afe44aba9b4d87cec3611ca: acquiring CA information for follower catalog manager: success
I20260808 16:56:23.743458 11184 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 52bafb3a7afe44aba9b4d87cec3611ca: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 16:56:23.834205 11252 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b90bb81c60a943daa7e1639eb2785c8c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 16:56:23.907976 11367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:23.905629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58202 (local address 127.8.24.61:40469)
0808 16:56:23.905726 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:23.905728 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:23.905780 (+ 52us) server_negotiation.cc:407] Connection header received
0808 16:56:23.905852 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:23.905853 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:23.905895 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:23.905957 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 16:56:23.906064 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:23.906582 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:23.907348 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:23.907535 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:23.907583 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:23.907713 (+ 130us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 16:56:23.907822 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 16:56:23.907883 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:23.907897 (+ 14us) server_negotiation.cc:299] Negotiation successful
0808 16:56:23.907915 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78,"thread_start_us":50,"threads_started":1}
I20260808 16:56:23.911662 11378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:23.909625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60016 (local address 127.8.24.60:39733)
0808 16:56:23.909736 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:23.909738 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:23.909746 (+ 8us) server_negotiation.cc:407] Connection header received
0808 16:56:23.909779 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:23.909780 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:23.909810 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:23.909870 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 16:56:23.909942 (+ 72us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:23.910454 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:23.911091 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:23.911281 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:23.911320 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:23.911423 (+ 103us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 16:56:23.911506 (+ 83us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 16:56:23.911560 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:23.911589 (+ 29us) server_negotiation.cc:299] Negotiation successful
0808 16:56:23.911606 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90,"thread_start_us":66,"threads_started":1}
I20260808 16:56:23.915572 11378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:23.913170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60024 (local address 127.8.24.60:39733)
0808 16:56:23.913203 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:23.913205 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:23.913333 (+ 128us) server_negotiation.cc:407] Connection header received
0808 16:56:23.913370 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:23.913371 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:23.913379 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:23.913417 (+ 38us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 16:56:23.913572 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:23.914254 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:23.914985 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:23.915212 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:23.915272 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:23.915368 (+ 96us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 16:56:23.915393 (+ 25us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 16:56:23.915440 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:23.915491 (+ 51us) server_negotiation.cc:299] Negotiation successful
0808 16:56:23.915510 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 16:56:23.915952 11367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:23.913127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58214 (local address 127.8.24.61:40469)
0808 16:56:23.913228 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:23.913229 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:23.913236 (+ 7us) server_negotiation.cc:407] Connection header received
0808 16:56:23.913259 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:23.913260 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:23.913267 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:23.913312 (+ 45us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 16:56:23.913417 (+ 105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:23.914144 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:23.915146 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:23.915411 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:23.915476 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:23.915627 (+ 151us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 16:56:23.915662 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 16:56:23.915848 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:23.915868 (+ 20us) server_negotiation.cc:299] Negotiation successful
0808 16:56:23.915888 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260808 16:56:23.916409 11391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:23.913043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58264 (local address 127.8.24.62:34315)
0808 16:56:23.913382 (+ 339us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:23.913383 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:23.913391 (+ 8us) server_negotiation.cc:407] Connection header received
0808 16:56:23.913432 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:23.913433 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:23.913477 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:23.913540 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 16:56:23.913646 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:23.914116 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:23.915516 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:23.915810 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:23.915869 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:23.916091 (+ 222us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 16:56:23.916181 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 16:56:23.916239 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:23.916310 (+ 71us) server_negotiation.cc:299] Negotiation successful
0808 16:56:23.916336 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":43,"threads_started":1}
I20260808 16:56:23.917784 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 10831
I20260808 16:56:23.942970 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 10944
I20260808 16:56:23.949965 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 11047
2026-08-08T16:56:23Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2871 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786207919384338-8288-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 16:56:24 dist-test-slave-6zbq krb5kdc[11398](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 16:56:24 dist-test-slave-6zbq krb5kdc[11398](info): set up 2 sockets
Aug 08 16:56:24 dist-test-slave-6zbq krb5kdc[11398](info): commencing operation
krb5kdc: starting...
W20260808 16:56:25.066602 8288 mini_kdc.cc:121] Time spent starting KDC: real 1.093s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786207919384338-8288-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786207919384338-8288-0/krb5kdc/test-user.keytab.
Aug 08 16:56:25 dist-test-slave-6zbq krb5kdc[11398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208185, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T16:56:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T16:56:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.24.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
I20260808 16:56:25.170593 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786207919384338-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786207919384338-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786207919384338-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:35167
--webserver_interface=127.8.24.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:39901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:35167,127.8.24.61:34323,127.8.24.60:45537
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:56:25.247794 11422 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 16:56:25.247978 11422 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:56:25.248003 11422 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:56:25.248026 11422 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:56:25.249110 11422 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 16:56:25.249151 11422 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:56:25.249171 11422 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:56:25.249188 11422 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 16:56:25.249203 11422 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 16:56:25.250694 11422 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:39901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786207919384338-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786207919384338-8288-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:35167,127.8.24.61:34323,127.8.24.60:45537
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:35167
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.24.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11422
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786207919384338-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:56:25.250924 11422 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:56:25.251116 11422 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:56:25.255340 11430 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:25.255355 11431 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:25.255375 11433 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:56:25.255573 11422 server_base.cc:1034] running on GCE node
I20260808 16:56:25.255806 11422 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:56:25.256039 11422 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:56:25.257179 11422 hybrid_clock.cc:648] HybridClock initialized: now 1786208185257163 us; error 27 us; skew 500 ppm
Aug 08 16:56:25 dist-test-slave-6zbq krb5kdc[11398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208185, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:56:25.259353 11422 init.cc:377] Logged in from keytab as kudu/127.8.24.62@KRBTEST.COM (short username kudu)
I20260808 16:56:25.259773 11422 webserver.cc:460] Webserver started at http://127.8.24.62:39079/ using document root <none> and password file <none>
I20260808 16:56:25.259970 11422 fs_manager.cc:359] Metadata directory not provided
I20260808 16:56:25.260015 11422 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:56:25.260133 11422 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:56:25.261037 11422 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786207919384338-8288-0/minicluster-data/master-0/data/instance:
uuid: "8fe5285f55be41679874a4555ee2b43f"
format_stamp: "Formatted at 2026-08-08 16:56:25 on dist-test-slave-6zbq"
server_key: "dad95834c1eca76a60f441190d657552"
server_key_iv: "05e90f9421b5b9f36832fd7af03edb31"
server_key_version: "encryptionkey@0"
I20260808 16:56:25.261365 11422 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786207919384338-8288-0/minicluster-data/master-0/wal/instance:
uuid: "8fe5285f55be41679874a4555ee2b43f"
format_stamp: "Formatted at 2026-08-08 16:56:25 on dist-test-slave-6zbq"
server_key: "dad95834c1eca76a60f441190d657552"
server_key_iv: "05e90f9421b5b9f36832fd7af03edb31"
server_key_version: "encryptionkey@0"
I20260808 16:56:25.262539 11422 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 16:56:25.263159 11441 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:25.263302 11422 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:25.263382 11422 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786207919384338-8288-0/minicluster-data/master-0/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786207919384338-8288-0/minicluster-data/master-0/wal
uuid: "8fe5285f55be41679874a4555ee2b43f"
format_stamp: "Formatted at 2026-08-08 16:56:25 on dist-test-slave-6zbq"
server_key: "dad95834c1eca76a60f441190d657552"
server_key_iv: "05e90f9421b5b9f36832fd7af03edb31"
server_key_version: "encryptionkey@0"
I20260808 16:56:25.263439 11422 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786207919384338-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786207919384338-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786207919384338-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:56:25.276460 11422 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:56:25.278074 11422 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:56:25.278206 11422 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:56:25.281817 11422 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.62:35167
I20260808 16:56:25.281845 11519 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.62:35167 every 8 connection(s)
I20260808 16:56:25.282126 11422 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786207919384338-8288-0/minicluster-data/master-0/data/info.pb
I20260808 16:56:25.282974 11520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:56:25.284262 11520 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35167 } has no permanent_uuid. Determining permanent_uuid...
I20260808 16:56:25.284473 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 11422
I20260808 16:56:25.284552 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786207919384338-8288-0/minicluster-data/master-0/wal/instance
I20260808 16:56:25.284745 8288 external_mini_cluster.cc:1428] Setting key f0f3721eebc68d404ade6b33274f5f78
Aug 08 16:56:25 dist-test-slave-6zbq krb5kdc[11398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208185, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.24.61@KRBTEST.COM; defaulting to no policy
I20260808 16:56:25.292785 11525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:25.284890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:35167 (local address 127.0.0.1:58678)
0808 16:56:25.285111 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:25.285149 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 16:56:25.285563 (+ 414us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:56:25.285806 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:56:25.285808 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:56:25.286042 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:56:25.286193 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:25.286197 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:25.287166 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:25.287167 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:25.287863 (+ 696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:25.287866 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:25.288325 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:25.288326 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:25.288388 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:25.288791 (+ 403us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:56:25.288799 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:56:25.289989 (+ 1190us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:56:25.291527 (+ 1538us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:56:25.291533 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:56:25.291535 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:56:25.291708 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:56:25.291921 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:56:25.291922 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:56:25.291922 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:56:25.292000 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:56:25.292292 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:56:25.292296 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:56:25.292354 (+ 58us) client_negotiation.cc:769] Sending connection context
0808 16:56:25.292432 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 16:56:25.292651 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"mutex_wait_us":91,"thread_start_us":81,"threads_started":1}
I20260808 16:56:25.292790 11523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:25.284897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58678 (local address 127.8.24.62:35167)
0808 16:56:25.285050 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:25.285058 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:25.285226 (+ 168us) server_negotiation.cc:407] Connection header received
0808 16:56:25.285701 (+ 475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:25.285704 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:25.285723 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:25.285797 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:25.286277 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:25.287081 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:25.287934 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:25.288237 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:25.288327 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:25.290062 (+ 1735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:25.290069 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:25.290069 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:56:25.290075 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:25.291457 (+ 1382us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:25.291775 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:25.291777 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:25.291778 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:25.291866 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:25.292052 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:25.292053 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:25.292053 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:25.292235 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:25.292301 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:25.292461 (+ 160us) server_negotiation.cc:299] Negotiation successful
0808 16:56:25.292570 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":75,"threads_started":1}
I20260808 16:56:25.293419 11520 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.24.61" port: 34323 } has no permanent_uuid. Determining permanent_uuid...
I20260808 16:56:25.293711 11525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:25.293593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.61:34323 (local address 127.0.0.1:39766)
0808 16:56:25.293655 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:25.293694 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.61:34323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260808 16:56:25.293823 11442 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.24.61:34323: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.61@KRBTEST.COM" created.
I20260808 16:56:25.294178 11525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:25.294092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.61:34323 (local address 127.0.0.1:39770)
0808 16:56:25.294133 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:25.294156 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.61:34323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260808 16:56:25.294255 11520 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.24.61:34323: Network error: Client connection negotiation failed: client connection to 127.8.24.61:34323: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.61.keytab.
Entry for principal kudu/127.8.24.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.61.keytab.
I20260808 16:56:25.306726 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786207919384338-8288-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786207919384338-8288-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786207919384338-8288-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786207919384338-8288-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.61:34323
--webserver_interface=127.8.24.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.61.keytab
--principal=kudu/127.8.24.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:39901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:35167,127.8.24.61:34323,127.8.24.60:45537
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 16:56:25.343420 11520 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.24.61" port: 34323 } attempt: 1
I20260808 16:56:25.343760 11525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:25.343663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.61:34323 (local address 127.0.0.1:39778)
0808 16:56:25.343708 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:25.343738 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.61:34323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260808 16:56:25.344044 11525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:25.343970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.61:34323 (local address 127.0.0.1:39784)
0808 16:56:25.344006 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:25.344029 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.61:34323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260808 16:56:25.344135 11520 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.24.61:34323: Network error: Client connection negotiation failed: client connection to 127.8.24.61:34323: connect: Connection refused (error 111)
W20260808 16:56:25.384555 11532 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 16:56:25.384744 11532 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:56:25.384768 11532 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:56:25.384794 11532 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:56:25.385875 11532 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 16:56:25.385914 11532 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:56:25.385932 11532 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:56:25.385948 11532 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 16:56:25.385965 11532 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 16:56:25.387434 11532 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:39901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786207919384338-8288-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786207919384338-8288-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:35167,127.8.24.61:34323,127.8.24.60:45537
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.61.keytab
--principal=kudu/127.8.24.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.61:34323
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786207919384338-8288-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.8.24.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11532
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786207919384338-8288-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:56:25.387663 11532 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:56:25.387868 11532 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 16:56:25.392099 11532 server_base.cc:1034] running on GCE node
W20260808 16:56:25.392074 11540 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:25.392066 11539 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:25.392074 11543 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:56:25.392478 11532 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:56:25.392743 11532 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:56:25.393877 11532 hybrid_clock.cc:648] HybridClock initialized: now 1786208185393863 us; error 27 us; skew 500 ppm
Aug 08 16:56:25 dist-test-slave-6zbq krb5kdc[11398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208185, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:56:25.396030 11532 init.cc:377] Logged in from keytab as kudu/127.8.24.61@KRBTEST.COM (short username kudu)
I20260808 16:56:25.396399 11532 webserver.cc:460] Webserver started at http://127.8.24.61:37557/ using document root <none> and password file <none>
I20260808 16:56:25.396587 11532 fs_manager.cc:359] Metadata directory not provided
I20260808 16:56:25.396641 11532 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:56:25.396756 11532 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:56:25.397658 11532 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786207919384338-8288-0/minicluster-data/master-1/data/instance:
uuid: "fbd81d6a909a42dfb0bf4641445ef7b8"
format_stamp: "Formatted at 2026-08-08 16:56:25 on dist-test-slave-6zbq"
server_key: "3b595ba6db1a040a6a1db0461ebd34c7"
server_key_iv: "570f0bf620d2f9a051078ebb03705a79"
server_key_version: "encryptionkey@0"
I20260808 16:56:25.397956 11532 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786207919384338-8288-0/minicluster-data/master-1/wal/instance:
uuid: "fbd81d6a909a42dfb0bf4641445ef7b8"
format_stamp: "Formatted at 2026-08-08 16:56:25 on dist-test-slave-6zbq"
server_key: "3b595ba6db1a040a6a1db0461ebd34c7"
server_key_iv: "570f0bf620d2f9a051078ebb03705a79"
server_key_version: "encryptionkey@0"
I20260808 16:56:25.399137 11532 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 16:56:25.399748 11552 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:25.399889 11532 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 16:56:25.399966 11532 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786207919384338-8288-0/minicluster-data/master-1/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786207919384338-8288-0/minicluster-data/master-1/wal
uuid: "fbd81d6a909a42dfb0bf4641445ef7b8"
format_stamp: "Formatted at 2026-08-08 16:56:25 on dist-test-slave-6zbq"
server_key: "3b595ba6db1a040a6a1db0461ebd34c7"
server_key_iv: "570f0bf620d2f9a051078ebb03705a79"
server_key_version: "encryptionkey@0"
I20260808 16:56:25.400022 11532 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786207919384338-8288-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786207919384338-8288-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786207919384338-8288-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:56:25.410651 11532 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:56:25.412142 11532 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:56:25.412281 11532 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:56:25.412269 11520 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.24.61" port: 34323 } attempt: 2
I20260808 16:56:25.412562 11525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:25.412446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.61:34323 (local address 127.0.0.1:39798)
0808 16:56:25.412496 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:25.412532 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.61:34323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260808 16:56:25.412925 11525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:25.412851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.61:34323 (local address 127.0.0.1:39804)
0808 16:56:25.412884 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:25.412907 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.61:34323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260808 16:56:25.413013 11520 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.24.61:34323: Network error: Client connection negotiation failed: client connection to 127.8.24.61:34323: connect: Connection refused (error 111)
I20260808 16:56:25.415942 11532 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.61:34323
I20260808 16:56:25.415971 11642 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.61:34323 every 8 connection(s)
I20260808 16:56:25.416240 11532 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786207919384338-8288-0/minicluster-data/master-1/data/info.pb
I20260808 16:56:25.417075 11643 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:56:25.418357 11643 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35167 } has no permanent_uuid. Determining permanent_uuid...
I20260808 16:56:25.420724 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 11532
I20260808 16:56:25.420821 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786207919384338-8288-0/minicluster-data/master-1/wal/instance
I20260808 16:56:25.421002 8288 external_mini_cluster.cc:1428] Setting key 1173718cf1302e2040379a6c34971eed
Aug 08 16:56:25 dist-test-slave-6zbq krb5kdc[11398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208185, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.61@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260808 16:56:25.426748 11523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:25.419006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58690 (local address 127.8.24.62:35167)
0808 16:56:25.419053 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:25.419055 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:25.419282 (+ 227us) server_negotiation.cc:407] Connection header received
0808 16:56:25.419677 (+ 395us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:25.419678 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:25.419688 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:25.419733 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:25.420229 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:25.420673 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:25.421979 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:25.422188 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:25.422241 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:25.424143 (+ 1902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:25.424145 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:25.424145 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:56:25.424152 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:25.425539 (+ 1387us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:25.425894 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:25.425895 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:25.425895 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:25.425956 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:25.426147 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:25.426148 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:25.426148 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:25.426277 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:25.426365 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:25.426576 (+ 211us) server_negotiation.cc:299] Negotiation successful
0808 16:56:25.426666 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 16:56:25.426780 11645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:25.418993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:35167 (local address 127.0.0.1:58690)
0808 16:56:25.419178 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:25.419211 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 16:56:25.419614 (+ 403us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:56:25.419778 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:56:25.419781 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:56:25.419984 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:56:25.420165 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:25.420169 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:25.420765 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:25.420766 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:25.421893 (+ 1127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:25.421896 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:25.422277 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:25.422278 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:25.422453 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:25.422875 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:56:25.422882 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:56:25.424057 (+ 1175us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:56:25.425615 (+ 1558us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:56:25.425622 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:56:25.425623 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:56:25.425826 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:56:25.426012 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:56:25.426013 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:56:25.426013 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:56:25.426095 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:56:25.426331 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:56:25.426334 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:56:25.426462 (+ 128us) client_negotiation.cc:769] Sending connection context
0808 16:56:25.426539 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 16:56:25.426647 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":76,"threads_started":1}
I20260808 16:56:25.427304 11643 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.24.61" port: 34323 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.8.24.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.60@KRBTEST.COM" created.
Aug 08 16:56:25 dist-test-slave-6zbq krb5kdc[11398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208185, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.61@KRBTEST.COM for kudu/127.8.24.61@KRBTEST.COM
I20260808 16:56:25.433595 11645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:25.427494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.61:34323 (local address 127.0.0.1:39808)
0808 16:56:25.427529 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:25.427537 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 16:56:25.427568 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:56:25.427808 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:56:25.427809 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:56:25.427930 (+ 121us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:56:25.427969 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:25.427971 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:25.428755 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:25.428756 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:25.429278 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:25.429280 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:25.429693 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:25.429694 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:25.429751 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:25.430065 (+ 314us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:56:25.430069 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:56:25.431233 (+ 1164us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:56:25.432699 (+ 1466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:56:25.432700 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:56:25.432701 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:56:25.432892 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:56:25.433100 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:56:25.433101 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:56:25.433101 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:56:25.433145 (+ 44us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:56:25.433409 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:56:25.433410 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:56:25.433441 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 16:56:25.433483 (+ 42us) client_negotiation.cc:240] Negotiation successful
0808 16:56:25.433542 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22}
I20260808 16:56:25.433710 11647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:25.427571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39808 (local address 127.8.24.61:34323)
0808 16:56:25.427701 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:25.427704 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:25.427717 (+ 13us) server_negotiation.cc:407] Connection header received
0808 16:56:25.427731 (+ 14us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:25.427732 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:25.427742 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:25.427796 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:25.428031 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:25.428688 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:25.429347 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:25.429628 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:25.429686 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:25.431307 (+ 1621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:25.431314 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:25.431314 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:56:25.431320 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:25.432604 (+ 1284us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:25.432977 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:25.432980 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:25.432981 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:25.433041 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:25.433197 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:25.433197 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:25.433198 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:25.433354 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:25.433429 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:25.433524 (+ 95us) server_negotiation.cc:299] Negotiation successful
0808 16:56:25.433606 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":83,"threads_started":1}
I20260808 16:56:25.434317 11643 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.24.60" port: 45537 } has no permanent_uuid. Determining permanent_uuid...
I20260808 16:56:25.434604 11645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:25.434510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.60:45537 (local address 127.0.0.1:35740)
0808 16:56:25.434552 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:25.434589 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.60:45537: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260808 16:56:25.434717 11554 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.24.60:45537: connect: Connection refused (error 111)
I20260808 16:56:25.435077 11645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:25.434994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.60:45537 (local address 127.0.0.1:35756)
0808 16:56:25.435033 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:25.435057 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.60:45537: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260808 16:56:25.435176 11643 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.24.60:45537: Network error: Client connection negotiation failed: client connection to 127.8.24.60:45537: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.60.keytab.
Entry for principal kudu/127.8.24.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.60.keytab.
I20260808 16:56:25.442976 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJykUMM/build/release/bin/kudu
/tmp/dist-test-taskJykUMM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786207919384338-8288-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786207919384338-8288-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786207919384338-8288-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786207919384338-8288-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.60:45537
--webserver_interface=127.8.24.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.60.keytab
--principal=kudu/127.8.24.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:39901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:35167,127.8.24.61:34323,127.8.24.60:45537
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786207919384338-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786207919384338-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786207919384338-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 16:56:25.484357 11643 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.24.60" port: 45537 } attempt: 1
I20260808 16:56:25.484689 11645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:25.484595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.60:45537 (local address 127.0.0.1:35764)
0808 16:56:25.484643 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:25.484670 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.60:45537: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260808 16:56:25.484968 11645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:25.484898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.60:45537 (local address 127.0.0.1:35768)
0808 16:56:25.484932 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:25.484954 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.60:45537: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260808 16:56:25.485049 11643 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.24.60:45537: Network error: Client connection negotiation failed: client connection to 127.8.24.60:45537: connect: Connection refused (error 111)
I20260808 16:56:25.504146 11520 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.24.61" port: 34323 } attempt: 3
Aug 08 16:56:25 dist-test-slave-6zbq krb5kdc[11398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208185, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for kudu/127.8.24.61@KRBTEST.COM
I20260808 16:56:25.510637 11525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:25.504362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.61:34323 (local address 127.0.0.1:39810)
0808 16:56:25.504405 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:25.504415 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 16:56:25.504457 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:56:25.504593 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:56:25.504594 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:56:25.504713 (+ 119us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:56:25.504767 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:25.504768 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:25.505600 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:25.505600 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:25.506084 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:25.506084 (+ 0us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:25.506562 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:25.506563 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:25.506626 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:25.506988 (+ 362us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:56:25.506992 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:56:25.508240 (+ 1248us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:56:25.509761 (+ 1521us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:56:25.509762 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:56:25.509762 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:56:25.509911 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:56:25.510094 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:56:25.510094 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:56:25.510094 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:56:25.510136 (+ 42us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:56:25.510417 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:56:25.510418 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:56:25.510458 (+ 40us) client_negotiation.cc:769] Sending connection context
0808 16:56:25.510509 (+ 51us) client_negotiation.cc:240] Negotiation successful
0808 16:56:25.510578 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260808 16:56:25.510782 11647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:25.504425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39810 (local address 127.8.24.61:34323)
0808 16:56:25.504474 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:25.504476 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:25.504486 (+ 10us) server_negotiation.cc:407] Connection header received
0808 16:56:25.504511 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:25.504512 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:25.504524 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:25.504589 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:25.504823 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:25.505527 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:25.506152 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:25.506485 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:25.506551 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:25.508324 (+ 1773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:25.508325 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:25.508325 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:56:25.508332 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:25.509683 (+ 1351us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:25.509975 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:25.509976 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:25.509977 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:25.510038 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:25.510187 (+ 149us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:25.510188 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:25.510188 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:25.510312 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:25.510424 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:25.510538 (+ 114us) server_negotiation.cc:299] Negotiation successful
0808 16:56:25.510684 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 16:56:25.511168 11520 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.24.60" port: 45537 } has no permanent_uuid. Determining permanent_uuid...
I20260808 16:56:25.511467 11525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:25.511372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.60:45537 (local address 127.0.0.1:35782)
0808 16:56:25.511399 (+ 27us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:25.511423 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.60:45537: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
I20260808 16:56:25.511709 11525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:25.511638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.60:45537 (local address 127.0.0.1:35786)
0808 16:56:25.511670 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:25.511691 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.60:45537: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260808 16:56:25.511786 11520 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.24.60:45537: Network error: Client connection negotiation failed: client connection to 127.8.24.60:45537: connect: Connection refused (error 111)
W20260808 16:56:25.521857 11654 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 16:56:25.522032 11654 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:56:25.522056 11654 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:56:25.522089 11654 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:56:25.523212 11654 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 16:56:25.523252 11654 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:56:25.523272 11654 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:56:25.523288 11654 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 16:56:25.523312 11654 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 16:56:25.524741 11654 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:39901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786207919384338-8288-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786207919384338-8288-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:35167,127.8.24.61:34323,127.8.24.60:45537
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786207919384338-8288-0/krb5kdc/kudu_127.8.24.60.keytab
--principal=kudu/127.8.24.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.60:45537
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786207919384338-8288-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.8.24.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11654
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786207919384338-8288-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:56:25.524967 11654 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:56:25.525158 11654 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:56:25.529312 11662 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:25.529312 11666 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:56:25.529330 11654 server_base.cc:1034] running on GCE node
W20260808 16:56:25.529332 11664 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:56:25.529716 11654 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:56:25.529990 11654 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:56:25.531131 11654 hybrid_clock.cc:648] HybridClock initialized: now 1786208185531111 us; error 32 us; skew 500 ppm
Aug 08 16:56:25 dist-test-slave-6zbq krb5kdc[11398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208185, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:56:25.533190 11654 init.cc:377] Logged in from keytab as kudu/127.8.24.60@KRBTEST.COM (short username kudu)
I20260808 16:56:25.533568 11654 webserver.cc:460] Webserver started at http://127.8.24.60:34855/ using document root <none> and password file <none>
I20260808 16:56:25.533761 11654 fs_manager.cc:359] Metadata directory not provided
I20260808 16:56:25.533806 11654 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:56:25.533922 11654 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:56:25.534839 11654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786207919384338-8288-0/minicluster-data/master-2/data/instance:
uuid: "5beb11cfde9c4abb9e30a8ab7d4f28b6"
format_stamp: "Formatted at 2026-08-08 16:56:25 on dist-test-slave-6zbq"
server_key: "a169e62ff9259bc4b8dbbabffd162c69"
server_key_iv: "c3aecd495b1c9d904e741cb421350c21"
server_key_version: "encryptionkey@0"
I20260808 16:56:25.535130 11654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786207919384338-8288-0/minicluster-data/master-2/wal/instance:
uuid: "5beb11cfde9c4abb9e30a8ab7d4f28b6"
format_stamp: "Formatted at 2026-08-08 16:56:25 on dist-test-slave-6zbq"
server_key: "a169e62ff9259bc4b8dbbabffd162c69"
server_key_iv: "c3aecd495b1c9d904e741cb421350c21"
server_key_version: "encryptionkey@0"
I20260808 16:56:25.536254 11654 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 16:56:25.536870 11675 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:25.537006 11654 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 16:56:25.537082 11654 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786207919384338-8288-0/minicluster-data/master-2/data,/tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786207919384338-8288-0/minicluster-data/master-2/wal
uuid: "5beb11cfde9c4abb9e30a8ab7d4f28b6"
format_stamp: "Formatted at 2026-08-08 16:56:25 on dist-test-slave-6zbq"
server_key: "a169e62ff9259bc4b8dbbabffd162c69"
server_key_iv: "c3aecd495b1c9d904e741cb421350c21"
server_key_version: "encryptionkey@0"
I20260808 16:56:25.537138 11654 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786207919384338-8288-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786207919384338-8288-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786207919384338-8288-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:56:25.542899 11520 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.24.60" port: 45537 } attempt: 1
I20260808 16:56:25.543159 11525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:25.543076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.60:45537 (local address 127.0.0.1:35800)
0808 16:56:25.543115 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:25.543141 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.60:45537: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260808 16:56:25.543422 11525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:25.543351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.60:45537 (local address 127.0.0.1:35806)
0808 16:56:25.543382 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:25.543403 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.60:45537: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
W20260808 16:56:25.543494 11520 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.24.60:45537: Network error: Client connection negotiation failed: client connection to 127.8.24.60:45537: connect: Connection refused (error 111)
I20260808 16:56:25.548990 11654 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:56:25.550819 11654 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:56:25.550938 11654 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:56:25.553203 11643 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.24.60" port: 45537 } attempt: 2
I20260808 16:56:25.553498 11645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:25.553399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.60:45537 (local address 127.0.0.1:35816)
0808 16:56:25.553447 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:25.553478 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.60:45537: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260808 16:56:25.553784 11645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:25.553704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.60:45537 (local address 127.0.0.1:35828)
0808 16:56:25.553739 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:25.553764 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.60:45537: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260808 16:56:25.553859 11643 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.24.60:45537: Network error: Client connection negotiation failed: client connection to 127.8.24.60:45537: connect: Connection refused (error 111)
I20260808 16:56:25.554595 11654 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.60:45537
I20260808 16:56:25.554622 11764 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.60:45537 every 8 connection(s)
I20260808 16:56:25.554889 11654 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786207919384338-8288-0/minicluster-data/master-2/data/info.pb
I20260808 16:56:25.555739 11765 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:56:25.556767 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJykUMM/build/release/bin/kudu as pid 11654
I20260808 16:56:25.556840 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJykUMM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786207919384338-8288-0/minicluster-data/master-2/wal/instance
I20260808 16:56:25.556990 8288 external_mini_cluster.cc:1428] Setting key 8b43cc05d30fb1ee92f19095d73c0643
I20260808 16:56:25.557075 11765 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35167 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 16:56:25 dist-test-slave-6zbq krb5kdc[11398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208185, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.60@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
Aug 08 16:56:25 dist-test-slave-6zbq krb5kdc[11398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208185, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.24.60@KRBTEST.COM
I20260808 16:56:25.565132 11523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:25.557669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58700 (local address 127.8.24.62:35167)
0808 16:56:25.557703 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:25.557705 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:25.557872 (+ 167us) server_negotiation.cc:407] Connection header received
0808 16:56:25.558081 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:25.558082 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:25.558092 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:25.558137 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:25.558704 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:25.559149 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:25.560124 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:25.560409 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:25.560477 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:25.562574 (+ 2097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:25.562576 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:25.562576 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:56:25.562583 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:25.563971 (+ 1388us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:25.564317 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:25.564319 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:25.564319 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:25.564374 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:25.564583 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:25.564584 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:25.564584 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:25.564690 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:25.564745 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:25.564981 (+ 236us) server_negotiation.cc:299] Negotiation successful
0808 16:56:25.565055 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260808 16:56:25.565275 11769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:25.557642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:35167 (local address 127.0.0.1:58700)
0808 16:56:25.557784 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:25.557812 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0808 16:56:25.558018 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:56:25.558154 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:56:25.558156 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:56:25.558442 (+ 286us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:56:25.558586 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:25.558590 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:25.559220 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:25.559221 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:25.560055 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:25.560057 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:25.560500 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:25.560501 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:25.560624 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:25.561037 (+ 413us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:56:25.561045 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:56:25.562497 (+ 1452us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:56:25.564035 (+ 1538us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:56:25.564042 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:56:25.564044 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:56:25.564251 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:56:25.564428 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:56:25.564429 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:56:25.564429 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:56:25.564528 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:56:25.564745 (+ 217us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:56:25.564752 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:56:25.564872 (+ 120us) client_negotiation.cc:769] Sending connection context
0808 16:56:25.564952 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 16:56:25.565085 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":74,"threads_started":1}
I20260808 16:56:25.565682 11768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:25.557386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35840 (local address 127.8.24.60:45537)
0808 16:56:25.557549 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:25.557557 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:25.557585 (+ 28us) server_negotiation.cc:407] Connection header received
0808 16:56:25.558012 (+ 427us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:25.558019 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:25.558039 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:25.558119 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:25.558753 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:25.559644 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:25.560477 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:25.560865 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:25.560918 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:25.563188 (+ 2270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:25.563195 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:25.563195 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:56:25.563201 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:25.564444 (+ 1243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:25.564759 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:25.564762 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:25.564762 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:25.564811 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:25.564969 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:25.564970 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:25.564970 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:25.565091 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:25.565202 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:25.565553 (+ 351us) server_negotiation.cc:299] Negotiation successful
0808 16:56:25.565619 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":81,"threads_started":1}
I20260808 16:56:25.565701 11765 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.24.61" port: 34323 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 16:56:25 dist-test-slave-6zbq krb5kdc[11398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208185, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.60@KRBTEST.COM for kudu/127.8.24.61@KRBTEST.COM
Aug 08 16:56:25 dist-test-slave-6zbq krb5kdc[11398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208185, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260808 16:56:25.572105 11647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:25.566016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39816 (local address 127.8.24.61:34323)
0808 16:56:25.566051 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:25.566053 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:25.566082 (+ 29us) server_negotiation.cc:407] Connection header received
0808 16:56:25.566102 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:25.566103 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:25.566111 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:25.566156 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:25.566425 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:25.567086 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:25.567903 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:25.568206 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:25.568285 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:25.569956 (+ 1671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:25.569957 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:25.569958 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:56:25.569965 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:25.571170 (+ 1205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:25.571472 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:25.571473 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:25.571473 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:25.571520 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:25.571688 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:25.571689 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:25.571689 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:25.571805 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:25.571865 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:25.571999 (+ 134us) server_negotiation.cc:299] Negotiation successful
0808 16:56:25.572055 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 16:56:25.572144 11769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:25.565973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.61:34323 (local address 127.0.0.1:39816)
0808 16:56:25.566009 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:25.566017 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 16:56:25.566047 (+ 30us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:56:25.566166 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:56:25.566166 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:56:25.566299 (+ 133us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:56:25.566359 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:25.566361 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:25.567145 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:25.567158 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:25.567839 (+ 681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:25.567840 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:25.568284 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:25.568284 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:25.568326 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:25.568646 (+ 320us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:56:25.568650 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:56:25.569879 (+ 1229us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:56:25.571238 (+ 1359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:56:25.571239 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:56:25.571240 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:56:25.571397 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:56:25.571571 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:56:25.571572 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:56:25.571572 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:56:25.571631 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:56:25.571862 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:56:25.571863 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:56:25.571903 (+ 40us) client_negotiation.cc:769] Sending connection context
0808 16:56:25.571981 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 16:56:25.572058 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260808 16:56:25.572430 11765 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.24.60" port: 45537 } has no permanent_uuid. Determining permanent_uuid...
I20260808 16:56:25.573514 11523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:25.566336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58706 (local address 127.8.24.62:35167)
0808 16:56:25.566392 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:25.566394 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:25.566403 (+ 9us) server_negotiation.cc:407] Connection header received
0808 16:56:25.566424 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:25.566425 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:25.566433 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:25.566479 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:25.566997 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:25.567641 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:25.568659 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:25.568933 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:25.569006 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:25.571078 (+ 2072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:25.571080 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:25.571080 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:56:25.571086 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:25.572491 (+ 1405us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:25.572838 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:25.572839 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:25.572839 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:25.572892 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:25.573063 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:25.573064 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:25.573064 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:25.573179 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:25.573334 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:25.573368 (+ 34us) server_negotiation.cc:299] Negotiation successful
0808 16:56:25.573438 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
Aug 08 16:56:25 dist-test-slave-6zbq krb5kdc[11398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208185, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.60@KRBTEST.COM for kudu/127.8.24.60@KRBTEST.COM
Aug 08 16:56:25 dist-test-slave-6zbq krb5kdc[11398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208185, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.24.61@KRBTEST.COM
I20260808 16:56:25.578475 11769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:25.572609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.60:45537 (local address 127.0.0.1:35842)
0808 16:56:25.572654 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:25.572665 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 16:56:25.572703 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:56:25.572816 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:56:25.572816 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:56:25.572932 (+ 116us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:56:25.572962 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:25.572963 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:25.573760 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:25.573761 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:25.574460 (+ 699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:25.574463 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:25.574787 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:25.574788 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:25.574825 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:25.575132 (+ 307us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:56:25.575137 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:56:25.576256 (+ 1119us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:56:25.577544 (+ 1288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:56:25.577544 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:56:25.577545 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:56:25.577709 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:56:25.577923 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:56:25.577923 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:56:25.577924 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:56:25.577984 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:56:25.578177 (+ 193us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:56:25.578177 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:56:25.578212 (+ 35us) client_negotiation.cc:769] Sending connection context
0808 16:56:25.578275 (+ 63us) client_negotiation.cc:240] Negotiation successful
0808 16:56:25.578367 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260808 16:56:25.578547 11768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:25.572652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35842 (local address 127.8.24.60:45537)
0808 16:56:25.572680 (+ 28us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:25.572682 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:25.572691 (+ 9us) server_negotiation.cc:407] Connection header received
0808 16:56:25.572759 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:25.572759 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:25.572766 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:25.572799 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:25.573023 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:25.573693 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:25.574532 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:25.574722 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:25.574782 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:25.576341 (+ 1559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:25.576342 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:25.576342 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:56:25.576347 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:25.577475 (+ 1128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:25.577786 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:25.577788 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:25.577788 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:25.577863 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:25.578035 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:25.578036 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:25.578036 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:25.578125 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:25.578176 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:25.578322 (+ 146us) server_negotiation.cc:299] Negotiation successful
0808 16:56:25.578420 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13}
I20260808 16:56:25.579660 11765 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5beb11cfde9c4abb9e30a8ab7d4f28b6: Bootstrap starting.
I20260808 16:56:25.580549 11765 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5beb11cfde9c4abb9e30a8ab7d4f28b6: Neither blocks nor log segments found. Creating new log.
I20260808 16:56:25.580595 11647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:25.574076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39822 (local address 127.8.24.61:34323)
0808 16:56:25.574107 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:25.574108 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:25.574116 (+ 8us) server_negotiation.cc:407] Connection header received
0808 16:56:25.574162 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:25.574162 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:25.574169 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:25.574208 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:25.574717 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:25.575164 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:25.575898 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:25.576148 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:25.576225 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:25.578263 (+ 2038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:25.578265 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:25.578265 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:56:25.578271 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:25.579645 (+ 1374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:25.579987 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:25.579988 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:25.579988 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:25.580033 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:25.580226 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:25.580227 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:25.580227 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:25.580312 (+ 85us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:25.580397 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:25.580495 (+ 98us) server_negotiation.cc:299] Negotiation successful
0808 16:56:25.580545 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260808 16:56:25.580816 11765 log.cc:824] T 00000000000000000000000000000000 P 5beb11cfde9c4abb9e30a8ab7d4f28b6: Log is configured to *not* fsync() on all Append() calls
I20260808 16:56:25.581497 11765 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5beb11cfde9c4abb9e30a8ab7d4f28b6: No bootstrap required, opened a new log
I20260808 16:56:25.581923 8288 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 16:56:25.582994 11765 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5beb11cfde9c4abb9e30a8ab7d4f28b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe5285f55be41679874a4555ee2b43f" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35167 } } peers { permanent_uuid: "fbd81d6a909a42dfb0bf4641445ef7b8" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 34323 } } peers { permanent_uuid: "5beb11cfde9c4abb9e30a8ab7d4f28b6" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 45537 } }
I20260808 16:56:25.583103 11765 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5beb11cfde9c4abb9e30a8ab7d4f28b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5beb11cfde9c4abb9e30a8ab7d4f28b6, State: Initialized, Role: FOLLOWER
I20260808 16:56:25.583177 11765 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5beb11cfde9c4abb9e30a8ab7d4f28b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe5285f55be41679874a4555ee2b43f" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35167 } } peers { permanent_uuid: "fbd81d6a909a42dfb0bf4641445ef7b8" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 34323 } } peers { permanent_uuid: "5beb11cfde9c4abb9e30a8ab7d4f28b6" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 45537 } }
I20260808 16:56:25.583458 11765 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5beb11cfde9c4abb9e30a8ab7d4f28b6 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 16:56:25.583429 11771 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5beb11cfde9c4abb9e30a8ab7d4f28b6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe5285f55be41679874a4555ee2b43f" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35167 } } peers { permanent_uuid: "fbd81d6a909a42dfb0bf4641445ef7b8" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 34323 } } peers { permanent_uuid: "5beb11cfde9c4abb9e30a8ab7d4f28b6" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 45537 } } }
I20260808 16:56:25.583523 11771 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5beb11cfde9c4abb9e30a8ab7d4f28b6 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 16:56:25.582881 11479 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "8fe5285f55be41679874a4555ee2b43f"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58706
W20260808 16:56:25.584930 11791 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5beb11cfde9c4abb9e30a8ab7d4f28b6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 16:56:25.584983 11791 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 16:56:25.585141 11654 master_runner.cc:186] Error getting master registration for 127.8.24.62:35167: OK, instance_id { permanent_uuid: "8fe5285f55be41679874a4555ee2b43f" instance_seqno: 1786208185280823 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 16:56:25.586594 11654 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 16:56:25.618618 11520 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.24.60" port: 45537 } attempt: 2
Aug 08 16:56:25 dist-test-slave-6zbq krb5kdc[11398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208185, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for kudu/127.8.24.60@KRBTEST.COM
I20260808 16:56:25.625811 11525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:25.618847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.60:45537 (local address 127.0.0.1:35844)
0808 16:56:25.618910 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:25.618919 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 16:56:25.618959 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:56:25.619082 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:56:25.619083 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:56:25.619241 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:56:25.619284 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:25.619285 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:25.620122 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:25.620123 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:25.620889 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:25.620891 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:25.621347 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:25.621348 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:25.621399 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:25.621798 (+ 399us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:56:25.621802 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:56:25.623216 (+ 1414us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:56:25.624821 (+ 1605us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:56:25.624822 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:56:25.624822 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:56:25.625037 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:56:25.625208 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:56:25.625209 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:56:25.625209 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:56:25.625271 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:56:25.625522 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:56:25.625523 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:56:25.625566 (+ 43us) client_negotiation.cc:769] Sending connection context
0808 16:56:25.625633 (+ 67us) client_negotiation.cc:240] Negotiation successful
0808 16:56:25.625718 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260808 16:56:25.625814 11768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:25.618929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35844 (local address 127.8.24.60:45537)
0808 16:56:25.618968 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:25.618970 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:25.618980 (+ 10us) server_negotiation.cc:407] Connection header received
0808 16:56:25.619002 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:25.619003 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:25.619011 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:25.619073 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:25.619345 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:25.620054 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:25.620962 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:25.621274 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:25.621337 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:25.623295 (+ 1958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:25.623297 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:25.623297 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:56:25.623304 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:25.624742 (+ 1438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:25.625103 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:25.625104 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:25.625104 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:25.625153 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:25.625324 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:25.625324 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:25.625325 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:25.625464 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:25.625524 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:25.625659 (+ 135us) server_negotiation.cc:299] Negotiation successful
0808 16:56:25.625730 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 16:56:25.627187 11520 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8fe5285f55be41679874a4555ee2b43f: Bootstrap starting.
I20260808 16:56:25.627856 11520 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8fe5285f55be41679874a4555ee2b43f: Neither blocks nor log segments found. Creating new log.
I20260808 16:56:25.628108 11520 log.cc:824] T 00000000000000000000000000000000 P 8fe5285f55be41679874a4555ee2b43f: Log is configured to *not* fsync() on all Append() calls
I20260808 16:56:25.628732 11520 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8fe5285f55be41679874a4555ee2b43f: No bootstrap required, opened a new log
I20260808 16:56:25.629662 11520 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8fe5285f55be41679874a4555ee2b43f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe5285f55be41679874a4555ee2b43f" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35167 } } peers { permanent_uuid: "fbd81d6a909a42dfb0bf4641445ef7b8" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 34323 } } peers { permanent_uuid: "5beb11cfde9c4abb9e30a8ab7d4f28b6" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 45537 } }
I20260808 16:56:25.629765 11520 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8fe5285f55be41679874a4555ee2b43f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fe5285f55be41679874a4555ee2b43f, State: Initialized, Role: FOLLOWER
I20260808 16:56:25.629835 11520 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8fe5285f55be41679874a4555ee2b43f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe5285f55be41679874a4555ee2b43f" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35167 } } peers { permanent_uuid: "fbd81d6a909a42dfb0bf4641445ef7b8" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 34323 } } peers { permanent_uuid: "5beb11cfde9c4abb9e30a8ab7d4f28b6" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 45537 } }
I20260808 16:56:25.630060 11795 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8fe5285f55be41679874a4555ee2b43f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe5285f55be41679874a4555ee2b43f" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35167 } } peers { permanent_uuid: "fbd81d6a909a42dfb0bf4641445ef7b8" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 34323 } } peers { permanent_uuid: "5beb11cfde9c4abb9e30a8ab7d4f28b6" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 45537 } } }
I20260808 16:56:25.630115 11520 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8fe5285f55be41679874a4555ee2b43f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 16:56:25.630137 11795 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8fe5285f55be41679874a4555ee2b43f [sys.catalog]: This master's current role is: FOLLOWER
I20260808 16:56:25.630244 11479 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8fe5285f55be41679874a4555ee2b43f [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260808 16:56:25.630362 11479 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8fe5285f55be41679874a4555ee2b43f [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:56:25.631131 11479 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8fe5285f55be41679874a4555ee2b43f [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe5285f55be41679874a4555ee2b43f" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35167 } } peers { permanent_uuid: "fbd81d6a909a42dfb0bf4641445ef7b8" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 34323 } } peers { permanent_uuid: "5beb11cfde9c4abb9e30a8ab7d4f28b6" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 45537 } }
W20260808 16:56:25.632134 11812 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8fe5285f55be41679874a4555ee2b43f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 16:56:25.632185 11812 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 16:56:25.632539 11422 master_runner.cc:186] Error getting master registration for 127.8.24.61:34323: OK, instance_id { permanent_uuid: "fbd81d6a909a42dfb0bf4641445ef7b8" instance_seqno: 1786208185414999 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 16:56:25.634738 11422 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 16:56:25.635097 11716 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8fe5285f55be41679874a4555ee2b43f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "5beb11cfde9c4abb9e30a8ab7d4f28b6"
I20260808 16:56:25.635213 11716 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5beb11cfde9c4abb9e30a8ab7d4f28b6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:56:25.635938 11716 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5beb11cfde9c4abb9e30a8ab7d4f28b6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fe5285f55be41679874a4555ee2b43f in term 1.
I20260808 16:56:25.636175 11442 leader_election.cc:302] T 00000000000000000000000000000000 P 8fe5285f55be41679874a4555ee2b43f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5beb11cfde9c4abb9e30a8ab7d4f28b6, 8fe5285f55be41679874a4555ee2b43f; no voters:
I20260808 16:56:25.636304 11795 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8fe5285f55be41679874a4555ee2b43f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:56:25.636488 11795 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8fe5285f55be41679874a4555ee2b43f [term 1 LEADER]: Becoming Leader. State: Replica: 8fe5285f55be41679874a4555ee2b43f, State: Running, Role: LEADER
I20260808 16:56:25.636579 11795 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8fe5285f55be41679874a4555ee2b43f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe5285f55be41679874a4555ee2b43f" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35167 } } peers { permanent_uuid: "fbd81d6a909a42dfb0bf4641445ef7b8" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 34323 } } peers { permanent_uuid: "5beb11cfde9c4abb9e30a8ab7d4f28b6" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 45537 } }
I20260808 16:56:25.636330 11479 leader_election.cc:288] T 00000000000000000000000000000000 P 8fe5285f55be41679874a4555ee2b43f [CANDIDATE]: Term 1 election: Requested vote from peers fbd81d6a909a42dfb0bf4641445ef7b8 (127.8.24.61:34323), 5beb11cfde9c4abb9e30a8ab7d4f28b6 (127.8.24.60:45537)
I20260808 16:56:25.637189 11815 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8fe5285f55be41679874a4555ee2b43f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8fe5285f55be41679874a4555ee2b43f. Latest consensus state: current_term: 1 leader_uuid: "8fe5285f55be41679874a4555ee2b43f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe5285f55be41679874a4555ee2b43f" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35167 } } peers { permanent_uuid: "fbd81d6a909a42dfb0bf4641445ef7b8" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 34323 } } peers { permanent_uuid: "5beb11cfde9c4abb9e30a8ab7d4f28b6" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 45537 } } }
I20260808 16:56:25.637279 11815 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8fe5285f55be41679874a4555ee2b43f [sys.catalog]: This master's current role is: LEADER
I20260808 16:56:25.637939 11819 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 16:56:25.638111 11819 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 16:56:25.644961 11643 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.24.60" port: 45537 } attempt: 3
I20260808 16:56:25.646001 11523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:25.638624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58710 (local address 127.8.24.62:35167)
0808 16:56:25.638668 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:25.638669 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:25.639027 (+ 358us) server_negotiation.cc:407] Connection header received
0808 16:56:25.639125 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:25.639126 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:25.639134 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:25.639163 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:25.639908 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:25.640595 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:25.641630 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:25.641927 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:25.641963 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:25.643382 (+ 1419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:25.643384 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:25.643384 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:56:25.643391 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:25.644868 (+ 1477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:25.645227 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:25.645228 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:25.645229 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:25.645279 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:25.645459 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:25.645460 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:25.645460 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:25.645569 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:25.645644 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:25.645849 (+ 205us) server_negotiation.cc:299] Negotiation successful
0808 16:56:25.645917 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 16:56:25.652005 11647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:25.638446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39838 (local address 127.8.24.61:34323)
0808 16:56:25.638484 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:25.638486 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:25.639442 (+ 956us) server_negotiation.cc:407] Connection header received
0808 16:56:25.639555 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:25.639556 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:25.639565 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:25.639602 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:25.640697 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:25.641415 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:25.643102 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:25.643430 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:25.643472 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:25.647094 (+ 3622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:25.647096 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:25.647096 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:56:25.647114 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:25.648821 (+ 1707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:25.649855 (+ 1034us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:25.649857 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:25.649857 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:25.649932 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:25.650270 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:25.650272 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:25.650272 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:25.650441 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:25.651508 (+ 1067us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:25.651834 (+ 326us) server_negotiation.cc:299] Negotiation successful
0808 16:56:25.651923 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 16:56:25.652899 11768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:25.639078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35858 (local address 127.8.24.60:45537)
0808 16:56:25.639109 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:25.639110 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:25.646201 (+ 7091us) server_negotiation.cc:407] Connection header received
0808 16:56:25.646296 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:25.646297 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:25.646307 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:25.646387 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:25.646936 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:25.647615 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:25.648569 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:25.648864 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:25.648900 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:25.650266 (+ 1366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:25.650267 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:25.650267 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:56:25.650274 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:25.651774 (+ 1500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:25.652142 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:25.652143 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:25.652143 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:25.652192 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:25.652376 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:25.652377 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:25.652377 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:25.652489 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:25.652549 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:25.652735 (+ 186us) server_negotiation.cc:299] Negotiation successful
0808 16:56:25.652815 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260808 16:56:25.655488 11716 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5beb11cfde9c4abb9e30a8ab7d4f28b6 [term 1 FOLLOWER]: Refusing update from remote peer 8fe5285f55be41679874a4555ee2b43f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:56:25.656121 11815 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8fe5285f55be41679874a4555ee2b43f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5beb11cfde9c4abb9e30a8ab7d4f28b6" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 45537 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
Aug 08 16:56:25 dist-test-slave-6zbq krb5kdc[11398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208185, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.61@KRBTEST.COM for kudu/127.8.24.60@KRBTEST.COM
I20260808 16:56:25.657429 11771 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5beb11cfde9c4abb9e30a8ab7d4f28b6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8fe5285f55be41679874a4555ee2b43f. Latest consensus state: current_term: 1 leader_uuid: "8fe5285f55be41679874a4555ee2b43f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe5285f55be41679874a4555ee2b43f" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35167 } } peers { permanent_uuid: "fbd81d6a909a42dfb0bf4641445ef7b8" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 34323 } } peers { permanent_uuid: "5beb11cfde9c4abb9e30a8ab7d4f28b6" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 45537 } } }
I20260808 16:56:25.657510 11771 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5beb11cfde9c4abb9e30a8ab7d4f28b6 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 16:56:25.659734 11645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:25.645214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.60:45537 (local address 127.0.0.1:35866)
0808 16:56:25.645311 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:25.645319 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 16:56:25.645353 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:56:25.653170 (+ 7817us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:56:25.653171 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:56:25.653314 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:56:25.653349 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:25.653351 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:25.654189 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:25.654190 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:25.654981 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:25.654984 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:25.655475 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:25.655476 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:25.655531 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:25.655873 (+ 342us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:56:25.655878 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:56:25.657146 (+ 1268us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:56:25.658797 (+ 1651us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:56:25.658798 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:56:25.658798 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:56:25.658990 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:56:25.659171 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:56:25.659172 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:56:25.659172 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:56:25.659223 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:56:25.659465 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:56:25.659465 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:56:25.659504 (+ 39us) client_negotiation.cc:769] Sending connection context
0808 16:56:25.659560 (+ 56us) client_negotiation.cc:240] Negotiation successful
0808 16:56:25.659645 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":82}
I20260808 16:56:25.659992 11768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:25.645409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35866 (local address 127.8.24.60:45537)
0808 16:56:25.652961 (+ 7552us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:25.652963 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:25.652969 (+ 6us) server_negotiation.cc:407] Connection header received
0808 16:56:25.652985 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:25.652985 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:25.652993 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:25.653041 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:25.653427 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:25.654098 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:25.655076 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:25.655355 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:25.655411 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:25.657233 (+ 1822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:25.657234 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:25.657234 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:56:25.657240 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:25.658699 (+ 1459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:25.659063 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:25.659063 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:25.659064 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:25.659112 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:25.659280 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:25.659281 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:25.659281 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:25.659390 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:25.659441 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:25.659846 (+ 405us) server_negotiation.cc:299] Negotiation successful
0808 16:56:25.659910 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7540,"thread_start_us":83,"threads_started":1}
I20260808 16:56:25.661556 11643 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fbd81d6a909a42dfb0bf4641445ef7b8: Bootstrap starting.
I20260808 16:56:25.662236 11643 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fbd81d6a909a42dfb0bf4641445ef7b8: Neither blocks nor log segments found. Creating new log.
I20260808 16:56:25.662573 11643 log.cc:824] T 00000000000000000000000000000000 P fbd81d6a909a42dfb0bf4641445ef7b8: Log is configured to *not* fsync() on all Append() calls
I20260808 16:56:25.664191 11643 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fbd81d6a909a42dfb0bf4641445ef7b8: No bootstrap required, opened a new log
I20260808 16:56:25.667541 11795 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8fe5285f55be41679874a4555ee2b43f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8fe5285f55be41679874a4555ee2b43f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe5285f55be41679874a4555ee2b43f" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35167 } } peers { permanent_uuid: "fbd81d6a909a42dfb0bf4641445ef7b8" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 34323 } } peers { permanent_uuid: "5beb11cfde9c4abb9e30a8ab7d4f28b6" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 45537 } } }
I20260808 16:56:25.667624 11795 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8fe5285f55be41679874a4555ee2b43f [sys.catalog]: This master's current role is: LEADER
I20260808 16:56:25.667789 11771 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5beb11cfde9c4abb9e30a8ab7d4f28b6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8fe5285f55be41679874a4555ee2b43f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe5285f55be41679874a4555ee2b43f" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35167 } } peers { permanent_uuid: "fbd81d6a909a42dfb0bf4641445ef7b8" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 34323 } } peers { permanent_uuid: "5beb11cfde9c4abb9e30a8ab7d4f28b6" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 45537 } } }
I20260808 16:56:25.667860 11771 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5beb11cfde9c4abb9e30a8ab7d4f28b6 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 16:56:25.666996 11643 raft_consensus.cc:348] T 00000000000000000000000000000000 P fbd81d6a909a42dfb0bf4641445ef7b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe5285f55be41679874a4555ee2b43f" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35167 } } peers { permanent_uuid: "fbd81d6a909a42dfb0bf4641445ef7b8" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 34323 } } peers { permanent_uuid: "5beb11cfde9c4abb9e30a8ab7d4f28b6" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 45537 } }
I20260808 16:56:25.668274 11643 raft_consensus.cc:729] T 00000000000000000000000000000000 P fbd81d6a909a42dfb0bf4641445ef7b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd81d6a909a42dfb0bf4641445ef7b8, State: Initialized, Role: FOLLOWER
I20260808 16:56:25.668476 11643 consensus_queue.cc:260] T 00000000000000000000000000000000 P fbd81d6a909a42dfb0bf4641445ef7b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe5285f55be41679874a4555ee2b43f" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35167 } } peers { permanent_uuid: "fbd81d6a909a42dfb0bf4641445ef7b8" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 34323 } } peers { permanent_uuid: "5beb11cfde9c4abb9e30a8ab7d4f28b6" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 45537 } }
I20260808 16:56:25.668923 11841 sys_catalog.cc:455] T 00000000000000000000000000000000 P fbd81d6a909a42dfb0bf4641445ef7b8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe5285f55be41679874a4555ee2b43f" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35167 } } peers { permanent_uuid: "fbd81d6a909a42dfb0bf4641445ef7b8" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 34323 } } peers { permanent_uuid: "5beb11cfde9c4abb9e30a8ab7d4f28b6" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 45537 } } }
I20260808 16:56:25.669159 11841 sys_catalog.cc:458] T 00000000000000000000000000000000 P fbd81d6a909a42dfb0bf4641445ef7b8 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 16:56:25.669087 11601 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8fe5285f55be41679874a4555ee2b43f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fbd81d6a909a42dfb0bf4641445ef7b8"
I20260808 16:56:25.669024 11643 sys_catalog.cc:564] T 00000000000000000000000000000000 P fbd81d6a909a42dfb0bf4641445ef7b8 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 16:56:25.669078 11598 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fbd81d6a909a42dfb0bf4641445ef7b8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:56:25.670167 11598 raft_consensus.cc:1261] T 00000000000000000000000000000000 P fbd81d6a909a42dfb0bf4641445ef7b8 [term 1 FOLLOWER]: Refusing update from remote peer 8fe5285f55be41679874a4555ee2b43f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:56:25.671008 11601 raft_consensus.cc:2454] T 00000000000000000000000000000000 P fbd81d6a909a42dfb0bf4641445ef7b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fe5285f55be41679874a4555ee2b43f in term 1.
I20260808 16:56:25.671691 11815 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8fe5285f55be41679874a4555ee2b43f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbd81d6a909a42dfb0bf4641445ef7b8" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 34323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:56:25.673206 11841 sys_catalog.cc:455] T 00000000000000000000000000000000 P fbd81d6a909a42dfb0bf4641445ef7b8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8fe5285f55be41679874a4555ee2b43f. Latest consensus state: current_term: 1 leader_uuid: "8fe5285f55be41679874a4555ee2b43f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe5285f55be41679874a4555ee2b43f" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35167 } } peers { permanent_uuid: "fbd81d6a909a42dfb0bf4641445ef7b8" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 34323 } } peers { permanent_uuid: "5beb11cfde9c4abb9e30a8ab7d4f28b6" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 45537 } } }
I20260808 16:56:25.673283 11849 mvcc.cc:205] Tried to move back new op lower bound from 7316308728442101760 to 7316308728368193536. Current Snapshot: MvccSnapshot[applied={T|T < 7316308728442101760}]
I20260808 16:56:25.673382 11841 sys_catalog.cc:458] T 00000000000000000000000000000000 P fbd81d6a909a42dfb0bf4641445ef7b8 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 16:56:25.673522 11841 sys_catalog.cc:455] T 00000000000000000000000000000000 P fbd81d6a909a42dfb0bf4641445ef7b8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8fe5285f55be41679874a4555ee2b43f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe5285f55be41679874a4555ee2b43f" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35167 } } peers { permanent_uuid: "fbd81d6a909a42dfb0bf4641445ef7b8" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 34323 } } peers { permanent_uuid: "5beb11cfde9c4abb9e30a8ab7d4f28b6" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 45537 } } }
I20260808 16:56:25.673633 11841 sys_catalog.cc:458] T 00000000000000000000000000000000 P fbd81d6a909a42dfb0bf4641445ef7b8 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 16:56:25.674465 11827 mvcc.cc:205] Tried to move back new op lower bound from 7316308728442101760 to 7316308728368193536. Current Snapshot: MvccSnapshot[applied={T|T < 7316308728442101760}]
W20260808 16:56:25.674733 11868 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fbd81d6a909a42dfb0bf4641445ef7b8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260808 16:56:25.674763 11819 catalog_manager.cc:1347] Generated new cluster ID: 30a4fda4f21d411e9cfff0aca19d6b16
W20260808 16:56:25.674803 11868 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 16:56:25.674808 11819 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 16:56:25.683375 11815 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8fe5285f55be41679874a4555ee2b43f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8fe5285f55be41679874a4555ee2b43f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe5285f55be41679874a4555ee2b43f" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35167 } } peers { permanent_uuid: "fbd81d6a909a42dfb0bf4641445ef7b8" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 34323 } } peers { permanent_uuid: "5beb11cfde9c4abb9e30a8ab7d4f28b6" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 45537 } } }
I20260808 16:56:25.683465 11815 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8fe5285f55be41679874a4555ee2b43f [sys.catalog]: This master's current role is: LEADER
I20260808 16:56:25.684089 11819 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 16:56:25.684779 11819 catalog_manager.cc:1504] Loading token signing keys...
I20260808 16:56:25.696125 11819 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8fe5285f55be41679874a4555ee2b43f: Generated new TSK 0
I20260808 16:56:25.696280 11819 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 16:56:25.723747 11647 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:56:25.722111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39844 (local address 127.8.24.61:34323)
0808 16:56:25.722151 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:25.722153 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:25.723103 (+ 950us) server_negotiation.cc:407] Connection header received
0808 16:56:25.723201 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:25.723202 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:25.723213 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:25.723242 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:25.723682 (+ 440us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39844: BlockingRecv error: recv got EOF from 127.0.0.1:39844 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260808 16:56:25.724434 11523 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:56:25.722368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58712 (local address 127.8.24.62:35167)
0808 16:56:25.722407 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:25.722409 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:25.723969 (+ 1560us) server_negotiation.cc:407] Connection header received
0808 16:56:25.724058 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:25.724059 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:25.724069 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:25.724095 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:25.724380 (+ 285us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58712: BlockingRecv error: recv got EOF from 127.0.0.1:58712 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260808 16:56:25.725070 11768 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:56:25.722695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35878 (local address 127.8.24.60:45537)
0808 16:56:25.722732 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:25.722733 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:25.724637 (+ 1904us) server_negotiation.cc:407] Connection header received
0808 16:56:25.724732 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:25.724732 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:25.724741 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:25.724765 (+ 24us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:25.725017 (+ 252us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35878: BlockingRecv error: recv got EOF from 127.0.0.1:35878 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260808 16:56:25.727123 11647 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:56:25.726425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39858 (local address 127.8.24.61:34323)
0808 16:56:25.726454 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:25.726456 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:25.726676 (+ 220us) server_negotiation.cc:407] Connection header received
0808 16:56:25.726767 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:25.726767 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:25.726783 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:25.726808 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:25.727071 (+ 263us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39858: BlockingRecv error: recv got EOF from 127.0.0.1:39858 (error 108)
Metrics: {"server-negotiator.queue_time_us":14}
W20260808 16:56:25.729933 11647 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:56:25.729254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39872 (local address 127.8.24.61:34323)
0808 16:56:25.729284 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:25.729286 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:25.729486 (+ 200us) server_negotiation.cc:407] Connection header received
0808 16:56:25.729562 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:25.729563 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:25.729572 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:25.729597 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:25.729893 (+ 296us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39872: BlockingRecv error: recv got EOF from 127.0.0.1:39872 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260808 16:56:25.735239 11647 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:56:25.734559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34194 (local address 127.8.24.61:34323)
0808 16:56:25.734590 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:25.734591 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:25.734805 (+ 214us) server_negotiation.cc:407] Connection header received
0808 16:56:25.734890 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:25.734891 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:25.734899 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:25.734924 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:25.735201 (+ 277us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34194: BlockingRecv error: recv got EOF from 127.0.0.1:34194 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260808 16:56:25.740916 11647 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:56:25.740238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34196 (local address 127.8.24.61:34323)
0808 16:56:25.740267 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:25.740269 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:25.740504 (+ 235us) server_negotiation.cc:407] Connection header received
0808 16:56:25.740578 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:25.740579 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:25.740587 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:25.740610 (+ 23us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:25.740881 (+ 271us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34196: BlockingRecv error: recv got EOF from 127.0.0.1:34196 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260808 16:56:25.750382 11647 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:56:25.749846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34206 (local address 127.8.24.61:34323)
0808 16:56:25.749879 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:25.749880 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:25.749987 (+ 107us) server_negotiation.cc:407] Connection header received
0808 16:56:25.750030 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:25.750030 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:25.750039 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:25.750084 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:25.750328 (+ 244us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34206: BlockingRecv error: recv got EOF from 127.0.0.1:34206 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260808 16:56:25.767685 11647 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:56:25.767187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34210 (local address 127.8.24.61:34323)
0808 16:56:25.767220 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:25.767222 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:25.767286 (+ 64us) server_negotiation.cc:407] Connection header received
0808 16:56:25.767341 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:25.767342 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:25.767351 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:25.767402 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:25.767644 (+ 242us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34210: BlockingRecv error: recv got EOF from 127.0.0.1:34210 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260808 16:56:25.801311 11647 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:56:25.800702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34212 (local address 127.8.24.61:34323)
0808 16:56:25.800743 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:25.800745 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:25.800849 (+ 104us) server_negotiation.cc:407] Connection header received
0808 16:56:25.800909 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:25.800910 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:25.800920 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:25.800969 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:25.801260 (+ 291us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34212: BlockingRecv error: recv got EOF from 127.0.0.1:34212 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260808 16:56:25.866962 11647 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:56:25.866365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34218 (local address 127.8.24.61:34323)
0808 16:56:25.866406 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:25.866408 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:25.866502 (+ 94us) server_negotiation.cc:407] Connection header received
0808 16:56:25.866546 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:25.866547 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:25.866556 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:25.866621 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:25.866900 (+ 279us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34218: BlockingRecv error: recv got EOF from 127.0.0.1:34218 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260808 16:56:25.996479 11647 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:56:25.995942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34220 (local address 127.8.24.61:34323)
0808 16:56:25.996009 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:25.996011 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:25.996068 (+ 57us) server_negotiation.cc:407] Connection header received
0808 16:56:25.996117 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:25.996118 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:25.996128 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:25.996175 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:25.996428 (+ 253us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34220: BlockingRecv error: recv got EOF from 127.0.0.1:34220 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260808 16:56:26.254169 11647 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:56:26.253579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34224 (local address 127.8.24.61:34323)
0808 16:56:26.253642 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:26.253645 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:26.253657 (+ 12us) server_negotiation.cc:407] Connection header received
0808 16:56:26.253736 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:26.253737 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:26.253756 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:26.253801 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:26.254122 (+ 321us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34224: BlockingRecv error: recv got EOF from 127.0.0.1:34224 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 16:56:26.585346 11791 catalog_manager.cc:1259] Loaded cluster ID: 30a4fda4f21d411e9cfff0aca19d6b16
I20260808 16:56:26.585407 11791 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 5beb11cfde9c4abb9e30a8ab7d4f28b6: loading cluster ID for follower catalog manager: success
I20260808 16:56:26.586156 11791 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 5beb11cfde9c4abb9e30a8ab7d4f28b6: acquiring CA information for follower catalog manager: success
I20260808 16:56:26.586467 11791 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 5beb11cfde9c4abb9e30a8ab7d4f28b6: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 16:56:26.675228 11868 catalog_manager.cc:1259] Loaded cluster ID: 30a4fda4f21d411e9cfff0aca19d6b16
I20260808 16:56:26.675297 11868 catalog_manager.cc:1552] T 00000000000000000000000000000000 P fbd81d6a909a42dfb0bf4641445ef7b8: loading cluster ID for follower catalog manager: success
I20260808 16:56:26.676103 11868 catalog_manager.cc:1574] T 00000000000000000000000000000000 P fbd81d6a909a42dfb0bf4641445ef7b8: acquiring CA information for follower catalog manager: success
I20260808 16:56:26.676414 11868 catalog_manager.cc:1602] T 00000000000000000000000000000000 P fbd81d6a909a42dfb0bf4641445ef7b8: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 16:56:26.769441 11965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:26.767209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34236 (local address 127.8.24.61:34323)
0808 16:56:26.767312 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:26.767314 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:26.767344 (+ 30us) server_negotiation.cc:407] Connection header received
0808 16:56:26.767413 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:26.767414 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:26.767452 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:26.767520 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 16:56:26.767643 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:26.768121 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:26.768806 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:26.768984 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:26.769024 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:26.769134 (+ 110us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 16:56:26.769260 (+ 126us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 16:56:26.769313 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:26.769330 (+ 17us) server_negotiation.cc:299] Negotiation successful
0808 16:56:26.769347 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86,"thread_start_us":58,"threads_started":1}
I20260808 16:56:26.773052 11975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:26.771088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43160 (local address 127.8.24.60:45537)
0808 16:56:26.771180 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:26.771182 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:26.771191 (+ 9us) server_negotiation.cc:407] Connection header received
0808 16:56:26.771228 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:26.771229 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:26.771271 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:26.771327 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 16:56:26.771399 (+ 72us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:26.771840 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:26.772491 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:26.772685 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:26.772738 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:26.772832 (+ 94us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 16:56:26.772917 (+ 85us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 16:56:26.772965 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:26.772980 (+ 15us) server_negotiation.cc:299] Negotiation successful
0808 16:56:26.772995 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73,"thread_start_us":50,"threads_started":1}
I20260808 16:56:26.777128 11990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:26.774512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40916 (local address 127.8.24.62:35167)
0808 16:56:26.774599 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:26.774600 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:26.774838 (+ 238us) server_negotiation.cc:407] Connection header received
0808 16:56:26.774899 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:26.774900 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:26.774948 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:26.775018 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 16:56:26.775116 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:26.775625 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:26.776322 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:26.776572 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:26.776629 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:26.776808 (+ 179us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 16:56:26.776897 (+ 89us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 16:56:26.776948 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:26.777028 (+ 80us) server_negotiation.cc:299] Negotiation successful
0808 16:56:26.777049 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69,"thread_start_us":46,"threads_started":1}
I20260808 16:56:26.777205 11965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:26.774508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34250 (local address 127.8.24.61:34323)
0808 16:56:26.774549 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:26.774551 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:26.774711 (+ 160us) server_negotiation.cc:407] Connection header received
0808 16:56:26.774765 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:26.774765 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:26.774776 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:26.774824 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 16:56:26.774957 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:26.775457 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:26.776469 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:26.776728 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:26.776780 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:26.776957 (+ 177us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 16:56:26.776986 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 16:56:26.777033 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:26.777125 (+ 92us) server_negotiation.cc:299] Negotiation successful
0808 16:56:26.777144 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 16:56:26.778096 11975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:26.774651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43168 (local address 127.8.24.60:45537)
0808 16:56:26.774701 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:26.774702 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:26.775515 (+ 813us) server_negotiation.cc:407] Connection header received
0808 16:56:26.775535 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:26.775535 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:26.775546 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:26.775583 (+ 37us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 16:56:26.775753 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:26.776428 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:26.777456 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:26.777721 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:26.777789 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:26.777901 (+ 112us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 16:56:26.777930 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 16:56:26.777974 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:26.778017 (+ 43us) server_negotiation.cc:299] Negotiation successful
0808 16:56:26.778035 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 16:56:26.778800 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 11422
I20260808 16:56:26.791965 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 11532
I20260808 16:56:26.800360 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJykUMM/build/release/bin/kudu with pid 11654
2026-08-08T16:56:26Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (2848 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (5720 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (267411 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST